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ABSTRACT 

This report synthesizes research on helping at-risk students 
meet academic standards. The studies chosen had to be published after 1984, 
be implemented in the United States, relate to K-12 students, directly assess 
students' academic achievement, address strategies that could be implemented 
in the regular classroom, and use strategies targeting low-achievement 
students. A total of 118 studies were sorted into six categories of classroom 
strategies: general instruction, cognitively oriented instruction, grouping 
structures ,• tutoring, peer tutoring, and computer-assisted instruction. 
Overall, results support the use of five of the strategies to help low- 
achieving students meet standards: cognitively oriented instruction, . 
heterogeneous grouping structures, tutoring, peer tutoring, and computer- 
assisted instruction. There are positive findings for the' effects of each 
strategy on the performance of low-achieving students, but in varying 
degrees, and with the exception of peer tutoring and computer-assisted 
instruction, only a minority of studies are considered high quality. Two 
appendices contain strategies to assist low-achieving students coding guide 
and meta-analysis methods. (Chapters contain references and annotated 
bibliographies.) (SM) 
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EXECUTIVE SUMMARY 



The No Child Left Behind Act of 2001 has focused attention on the responsibility of schools and 
districts to help all children achieve at high levels. However, it is in classrooms that children receive the 
instruction and assistance that they need to achieve standards. For this reason, teachers need access to 
practices that target children who are low performing or at-risk of failure. Thus, this synthesis addressed 
the following research problem: What are effective instructional strategies that can be used in classrooms 
to assist low-achieving students? 

The synthesis methods used to create this report follow established guidelines for conducting 
research syntheses in education and the social sciences. Students who are not meeting standards were 
defined as K-12 students who are low performing on an academic assessment or who are at risk for low 
performance based on factors such as high poverty. A strategy was defined as a practice or intervention 
designed to help low-achieving students at the classroom level. Based on these operational definitions and 
the research problem, the following inclusion criteria were established: 

• The study concerned K-12 students. 

• The study reported a direct assessment of students’ academic achievement. 

• The study concerned a strategy that could be implemented in the regular classroom. 

• The study concerned a strategy that targeted low-achieving students. 

• The study was published in or after 1985. 

• The study was implemented in the United States. 

To provide the broadest array of findings, the synthesis examined both published and 
unpublished studies and qualitative as well as quantitative research. Research on commercial programs 
was included if evaluation results were not previously reported in detail. Initial literature searches 
produced 4,291 citations and 727 articles fi'om which 118 research studies were identified that met the 
inclusion criteria. 

An instrument was developed for coding the content and research quality of each study. Studies 
were assigned a quality index of high, medium, or low based on characteristics related to research 
validity. To better analyze and communicate the findings, studies were sorted into six categories of 
classroom strategies. 

The General Instruction category examined 15 studies that met the inclusion criteria. There were 
10 studies of behaviorist strategies that utilized either explicit or repeated instruction and five studies of 
constructivist strategies that emphasized meaning and student understanding or balanced these with 
explicit instruction. Half of the behaviorist studies produced positive results on student achievement, but 
only one study was coded as high in research quality. All of the constructivist studies were coded as 
having medium or high quality, but only two of the studies — those of constructivist meaning-based 
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instruction — produced positive results on student achievement. The synthesis results on general 
instruction were insufficient to make recommendations. Because the schism between behaviorist and 
constructivist instruction is philosophically based, practitioners should consider how each approach views 
low-achieving students and the role of the classroom teacher. 

The category of Cognitively Oriented Instruction examined 15 studies that met the inclusion 
criteria. There were nine studies of cognitively oriented instruction in reading, including two studies 
coded as high quality and four coded as medium quality. There were positive effects on student 
achievement in reading in studies where instruction addressed task planning/preparation, and how-to 
strategies such as summarizing were taught. There were four studies of co^itively oriented instruction in 
writing and oral language, all of which met the criteria for medium- or high-quality research. Positive 
effects on achievement occurred when students were taught strategies for task analysis and preparation 
and had opportunities for practice, feedback, and discussion about the writing task. One medium- and one 
high-quality study of cognitively oriented instruction in mathematics found positive results on student 
achievement and highlighted the benefits of using meaningful, relevant problems. Due to the small 
number of studies and the lack of research on long-term implementation, only preliminary conclusions 
can be made that cognitively oriented instruction is an effective strategy for assisting low-achieving 
students. However, the reviewed studies suggest to practitioners some indicators of effective cognitively 
oriented instruction. 

The category of Grouping Structures examined 18 studies that met the inclusion criteria. There 
were 10 studies of grouping students across ability levels, four of which were coded as high quality. Nine 
of the studies found positive results on student achievement using cooperative learning approaches. 
Students were in middle grades or higher. There were eight studies of grouping students by ability levels, 
and all employed either direct instruction or teacher-led discussions. Five of these were implemented with 
lower elementary students. All but one of the eight studies found positive effects on student achievement, 
and the exception was the one study coded as high quality. Overall, the synthesis of research on grouping 
structures resulted in preliminary conclusions that cooperative grouping across ability levels is an 
effective strategy for low-achieving students. The lack of high-quality research on grouping by ability 
levels precluded recommendations regarding this practice. Practitioners should be aware of factors that 
can make student grouping successful, including presenting students with a clearly defined task, 
facilitating student helping and questioning behaviors, and providing teacher training in group facilitation. 

The Tutoring category examined 23 studies that met the inclusion criteria. There were five studies 
of professional tutors, such as teachers. Four of the studies, including the one high- quality study, found 
positive effects of tutoring on student achievement. All four examined tutoring of K*~l students in reading 
and utilized diagnostic-prescriptive approaches. There were 13 studies of volunteer tutoring of students in 
grades K-12, primarily in reading. Eight of the studies found positive results on student achievement, 
including three of the four high-quality studies. The successful volunteer tutoring programs utilized 
purposeful instructional frameworks to guide the tutoring. Recruiting and retaining volunteers were 
highlighted as logistical concerns. There were five studies of tutoring students in grades 1-9 in literacy 
where the tutors were older students. None of the studies was coded as high quality. Four of the studies 
found positive effects of cross-age student tutoring on reading achievement. Tutor training was an 
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important factor in cross-age tutoring as well as in professional and volunteer tutoring. Overall, the 
synthesis results for the category of Tutoring supported a preliminary conclusion that programs utilizing 
different types of tutors can result in positive effects on the achievement of low-performing students. 
Practitioners should be aware of the characteristics of effective tutoring programs including program 
monitoring, a purposeful instructional framework to guide the tutoring, and the attention given to the 
diagnostic and prescriptive interaction. 

The Peer Tutoring category examined 30 studies within three types of peer tutoring that met the 
inclusion criteria. In classwide peer tutoring, typically pairs of students instruct and assist each other in 
learning basic information in a variety of subjects from two to five days each week. There were 10 studies 
of classwide peer tutoring, most of which occurred in elementary grades and in literacy-related subjects. 
All six studies for which the direction of results could be determined produced positive effects on the 
achievement of low-performing students. Two of the studies were high-quality longitudinal studies. There 
were eight studies of Peer-Assisted Learning Strategies (PALS), a peer tutoring protocol specifically 
designed to engage students in strategic reading activities. All eight studies of PALS were coded as high 
quality and concerned elementary students. Six studies included learning-disabled students. All the 
studies resulted in positive effects of PALS on reading achievement of low-performing students. There 
were six studies of Reciprocal Peer Tutoring, a strategy designed to improve mathematics achievement in 
elementary students. This method employs alternating student tutor and teaching roles and reinforcement 
for learning. All of the five studies that provided sufficient information concerning the results were high 
quality, random assignment studies; all found positive effects in the mathematics achievement of low- 
performing students. There was a final subcategory of six studies that included a mix of low-achieving 
students in grades 1-10 in a variety of subjects. Only three of these studies provided sufficient 
information about the direction of results, and of these, two found positive effects on achievement. 
Overall, the results of the synthesis of studies on peer tutoring supported preliminary conclusions that the 
strategy is effective for low-achieving and learning disabled elementary students in basic skills for both 
literacy and mathematics. Practitioners should be aware of the need for training of both teachers and 
students, complex classroom logistics, and careful teacher monitoring of tutoring behaviors and activities. 

The category of Computer- Assisted Instruction examined 2 1 studies in which the computer was 
used to improve the achievement of low-performing students. Seventeen of the studies provided sufficient 
quantitative information to conduct a meta-analysis. Of these, eight studies concerned students in grades 
1-6, and the remainder concerned students in grades 7-12. The subjects were literacy and mathematics. 
The meta-analysis produced an overall statistically significant effect size of .37, which means that 
students who received computer-assisted instruction scored an average of 14 percentile points higher than 
those who did not receive this strategy. Additional analyses found that computer-assisted instruction had a 
significantly larger effect on achievement in mathematics (.57) than in literacy (.16). There were no 
significant differences between the effect sizes of high- versus low-quality research studies. The 
statistically significant effect size from the meta-analysis provided conclusive evidence that computer- 
assisted instruction is an effective strategy for improving achievement of low-performing students, 
particularly in mathematics. Practitioners should note the advantages of computer-assisted instruction, 
which include individualized learning, immediate feedback, and the potential for increased student 
autonomy and motivation to learn. 
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Overall, this research synthesis provided evidence that supports the use of five strategies to help 
low-achieving students meet standards: cognitively oriented instruction, heterogeneous grouping 
structures, tutoring, peer tutoring, and computer-assisted instruction. The authors cited the need for more 
high-quality research studies and studies that build on previous findings in order to accumulate a body of 
knowledge about specific interventions. While practitioner guidance was not a goal of the synthesis, 
implementation issues associated with each strategy were identified. Practitioners should consult the 
resources cited at the end of each chapter for additional implementation guidance. 



PREFACE 



The No Child Left Behind Act of 2001 places responsibility on states to ensure that their schools 
help all students succeed, that is, to achieve standards, as the name of the Act reflects. In 12 years, all 
students must perform at a “proficient” level on state tests. Therefore, states are under more pressure than 
ever before to raise student achievement. Furthermore, states must monitor whether schools are making 
adequate yearly progress as measured by assessments aligned with state standards. This context has 
created the need to identify strategies that can assist students who are not meeting standards. The purpose 
of this report is to review and synthesize the research on such classroom practices. 

In 2001, Mid-continent Research for Education and Learning (McREL) published a research 
synthesis on standards in classroom practice (Apthorp et al.). That report provided narrative reviews of 
research on standards-based instruction in literacy and mathematics and the practices and policies needed 
for professional development and school organizations in a standards-based education system. In 2002, 
we emphasize specific classroom practices that can help students who are struggling to meet the 
standards. This year we have conducted a research synthesis (see frontispiece) on the effectiveness of 
strategies designed to assist low-achieving students during the school day. In 2003, we intend to 
synthesize the research on strategies to assist low-achieving students outside the school day. 

The goals for the current synthesis are the following: 

1. Based on an extensive and representative collection of research studies, to identify 
evidence strategies designed to assist low-achieving students during the school day. 

2. To assess the validity and reliability of the findings about the strategies according to 
the quality of the research. 

3. To describe research-based strategies for researchers and practitioners based on our 
confidence in the findings. 

Our approach was to examine systematically the research about classroom strategies and to 
consider in our analysis the degree to which the research meets standards of rigor. As the Director of the 
Institute of Education Sciences has indicated, all education research is not of equal merit, yet decisions in 
education should be based on sound research evidence (Whitehurst, 2001). Our methods reflect this view 
and follow established guidelines for conducting research syntheses. 

As educators face new challenges to ensure that low-performing student achieve standards, we 
must find and adopt practices that are substantiated through rigorous research. These practices are more 
than merely research based in that their effectiveness has been demonstrated through testing in the field. 
One way to identify evidence-based practices is through a systematic research synthesis; a strategy 
recently given visibility by the U.S. Department of Education when it created the What Works 
Clearinghouse to conduct such syntheses. 

Each of the federally funded Regional Education Laboratories has also been asked to conduct an 
annual research synthesis. A research synthesis is not a practitioner-oriented, how-to paper. The measure 
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of its success is whether or not it includes a systematically collected comprehensive set of research on a 
practice, a thorough assessment of the validity of the relevant research, and a clear statement of what can 
be learned from that research about the effectiveness of the practice. It is also valuable to learn proof is 
lacking. The conduct of a successful research synthesis requires the implementation of a rigorous 
methodology that includes 

• exhaustively searching for studies in published and unpublished literature (to find 
both positive and negative results), 

• assessing each study for validity to insure reliability of results, rejecting studies of 
insufficient quality, and 

• deriving findings about the body of evidence in existence about the practice. 

A synthesis answers researchers’ questions about the adequacy of current research and highlights 
areas where the available research is insufficient. Practitioners learn to what degree a practice has been 
proven to work, but must determine for themselves the specifics of implementation in their settings. 

We have organized this research synthesis into eight chapters. Chapter 1 provides a general 
overview of research on strategies to help low-achieving students and describes the methods that we used 
to search the literature and to select and code research studies. Chapters 2 through 7 review and 
synthesize research on each of the six general strategies that emerged from our literature searches. In 
Chapter 8, we summarize the findings across the six strategies and provide general conclusions. 

The primary audience for this document includes education researchers and state administrators 
who have a general understanding of research methods and scientifically based evidence. The secondary 
audience includes district administrators and staff developers who have some background in research. 
Given the goals of this synthesis, we designed the document to be a hybrid of a highly technical synthesis 
and a report that can readily assist state departments of education and district personnel. We used 
systematic methods to review and analyze the content and quality of the studies based on research 
standards. To serve different types of audiences, we used both technical and non-technical language and 
description. Finally, although K-12 practitioners are not the primary audience for this document', each 
chapter concludes with an annotated bibliography and a section for practitioners that highlights the 
features of the strategy, research findings about the strategy, issues that occur when implementing the 
strategy, and resources on strategy implementation 



' McREL intends to develop a follow-up document for practitioners that addresses strategies to help low- 
achieving students to meet standards. 
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CHAPTER. 1 



BACKGROUND AND METHODS 

Patricia A. Lauer 

Research Problem 

The No Child Left Behind Act of 2001, which reauthorized the Elementary and Secondary 
Education Act, has focused attention on the responsibility of schools and districts to help all children 
achieve at high levels. The underlying assumption of the Act is that states will continue to implement 
standards for student learning and will measure students’ progress through assessments that are aligned 
with these standards. Ultimately, it is in classrooms that children will receive the instruction and 
assistance that they need to achieve standards. In Marzano’s (2000) analysis of the influences of school, 
teacher, and student variables on student achievement, variables related to teaching (e.g., instructional 
practices) accounted for 13.3 percent of the variation, while school-level variables (e.g., school climate) 
accounted for 6.6 percent. Student variables (e.g., socio-economic status) had the greatest impact at 80 
percent. These results suggest that teachers’ classroom practices have the potential to influence student 
outcomes to a greater degree than do school practices. It is important for teachers to have access to the 
most effective practices that target children who are at-risk of failure because of the influence of student 
variables. Thus, our synthesis addresses the following research problem: What are effective strategies that 
can be used to assist low-achieving students? 

Research Context 

Much of the prior research on ways to assist low-achieving students has centered around the 
notion of “at-risk.” Although researchers define at-risk in different ways, the term generally refers to 
students who are at risk for school failure (Slavin & Madden, 1989). Analyses of large-scale data 
collections have found that lower student achievement and school dropout rates are associated with 
certain student characteristics, including low socio-economic status (SES), racial or ethnic minority 
background, single-parent family, mother with low education, and limited proficiency in English (Miller, 
1993). 



Waxman, Padron, and Arnold (2001) described instructional practices that are effective for at-risk 
students through a review of research related to five areas: cognitively guided instruction, culturally 
responsive instruction, technology-enriched instruction, cooperative learning, and instructional 
conversation (student-centered discussion). Howe and Kasten (1992) and McLaughlin and Vacha (1992) 
wrote similar reports on practices and programs to assist at-risk students, and Means, Chelemer, and 
Knapp (1991) presented six instructional models for teaching at-risk students. These descriptive 
summaries provide helpful overviews and recommendations about strategies to help low-achieving 
students, the topic of the current synthesis. However, these prior reviews did not attempt to synthesize this 
research by examining research quality and research results and presenting evidence of the effectiveness 
of the strategies based on the body of studies. 
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Slavin and Madden (1989) conducted a best-evidence synthesis of classroom programs for at-risk 
students. With this approach, only studies that meet certain pre-established criteria are analyzed to reach 
conclusions. The authors cited four types of programs that were supported by evidence of effectiveness 
using their criteria: continuous progress programs in which students progress at their own pace and are 
taught in small groups, cooperative learning, remedial tutoring, and computer-assisted instruction. The 
current synthesis updates Slavin and Madden’s report and identifies additional strategies. 

Another source of research on assistance for low-achieving students are evaluations of strategies 
and programs funded by Title 1 (formerly called Chapter 1) of the No Child Left Behind Act. Title 1 
provides federal funds to districts and states to improve education opportunities and outcomes for 
students who are at risk of failure due to low income. Borman and D’Agostino (1996) conducted a meta- 
analysis of federal evaluations on the relationship between programs funded by Title 1 and student 
achievement. The authors found “a modest overall impact” (p. 309) of Title 1 -funded programs on student 
achievement. Rowan and Guthrie’s (1989) study of Title 1 found large variations across schools in the 
content and quality of Title 1 instruction compared to the content of regular instruction not supported by 
Title 1. Due to this variation, the authors were unable to draw conclusions about effective Title 1- 
supported strategies. Other research has examined Title 1 comprehensive school reform models (Wang, 
Haertel, & Walberg, 1997; Fashola & Slavin, 1998). However, most of this research does not isolate the 
specific classroom strategies that are key to the success of low-achieving students in the programs. 

In summary, reviews of research on assistance for low-achieving students have emphasized 
studies of strategies and programs that can help students overcome factors associated with risk of 
educational failure. To update and assess the quality of this research, we conducted a research synthesis of 
studies completed since 1985 and coded research quality as a mediator of the findings. We conducted an 
extensive search for studies, so we could examine the broad array of findings, both positive and negative. 
Research on at-risk students in whole-school reform models tends not to identify the specific strategies in 
a program that result in improved achievement. Thus, we focused on studies that isolate the effects of 
specific strategies or programs. Our synthesis fills the need for information about research-based 
strategies that schools and teachers can use to help their low-achieving students meet standards. 

Methods 

As described in this section, we used systematic methods to conduct a review of research on 
strategies designed to assist low-achieving students. Where the research supported it, we used meta- 
analytic techniques. Otherwise, the reviews are narrative in form. In conducting the synthesis, we drew on 
guidance from previous researchers who have published on synthesis methodology (Cooper, Charlton, 
Valentine, & Muhlenbruck, 2000; Shanahan, 2000) and particularly from the work of Harris Cooper 
(1998). Cooper is directing a sub-contract for the U.S. Department of Education’s newly funded What 
Works Clearinghouse, which will publish research reviews on the effectiveness of programs and 
strategies intended to enhance student achievement (U.S. Department of Education, 2002). 
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Literature Searches 



The goal of our search methods was to accurately and comprehensively represent the research on 
the synthesis topic. To operationalize the topic, we defined students who are not meeting standards as K- 
12 students who are (a) low performing on an academic assessment, or (b) at risk for low performance 
based on factors such as high poverty. We defined a strategy as a practice or intervention designed to help 
low-achieving students. A strategy also could be a program, but we excluded school-wide programs 
because the components that are most responsible for improved achievement cannot be isolated. For 
example, our definition includes specific strategies that are funded by Title 1, such as tutoring, but 
excludes Title 1 comprehensive school reform models. Based on our operational definitions and goals, we 
established the following initial criteria for including a study: 

• The study had to concern K-12 students. 

• The study had to include some type of direct assessment of students’ academic 
achievement, such as a classroom or standardized test. Measures of dropout and 
student motivation did not qualify as measures of academic achievement. 

• The study had to be published or reported in or after 1985. (We chose this date as the 
approximate start of the standards movement in the United States.) 

• The study had to be implemented in the United States. 

Using 15 terms identified through a preliminary review of the research literature, we conducted 
an initial search of ERIC and Psychinfo related to strategies for low-achieving or at-risk students. Search 
terms included terms such as “extended time” and “tutoring.” Initial searches indicated that strategies 
were either implemented during the school day, such as class-wide peer tutoring, or implemented outside 
the school day, such as summer school. Strategies implemented outside the school day are influenced by a 
different set of resource decisions, such as the need for additional instructional time and the funds to 
support additional instruction. We therefore decided to limit this synthesis to strategies that are 
implemented during the school day, and to address the outside-the-school day strategies following the 
completion of this study. 

Abstracts from our initial search were the bases of additional searches of ERIC and Psych Info for 
the years 1985 through 2002. For these searches, we separately combined the terms “low achievement” 
and “at risk” with 20 terms that our initial searches indicated were used for strategies to assist low- 
achieving students during the school day, for example, “remediation and low achievement.” These new 
searches produced 4,291 citations. After reading abstracts for these citations and determining which 
studies met initial criteria, we obtained 727 articles on research studies, reviews, and syntheses. We 
examined these articles and established the following additional inclusion criteria: 

• The study concerned a strategy that is implemented in the context of a regular 
classroom setting. For example, we included studies of strategies to assist special 
education students if the classroom teacher implemented the strategy in an inclusion 
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setting. However, we excluded studies of strategies that special education teachers 
implemented to assist special education students in a resource classroom. We used 
the same logic to include or exclude studies of strategies for students who are limited 
in English proficiency. 

• The study included an examination of the effectiveness of a strategy for low- 
achieving students or students at risk for low achievement. We excluded studies if 
data analyses related to achievement were incidental and not related to the purpose of 
the study. 

• The study examined a commercial program for which evaluation results have not 
previously been described in sufficient detail. We conducted an additional literature 
search for each excluded commercial program. If this additional search confirmed the 
existence of sufficient previously reported research, we excluded the program from 
the synthesis.^ For example, we excluded studies of Reading Recovery because of the 
breadth of research reported on this commercial program. However, our synthesis 
includes strategies such as tutoring that may be components of such commercial 
programs. 

There are some additional review criteria that require explanation. To provide the broadest array 
of findings, we reviewed unpublished studies such as evaluation reports, conference presentations, and 
dissertations for inclusion. A benefit of this approach is that it helps researchers avoid the null hypothesis 
problem (Cooper, 1998) whereby studies that found no effect are excluded from a synthesis because they 
were not published. This problem tends to bias a synthesis in favor of finding positive results. We also 
reviewed qualitative as well as quantitative studies. The use of qualitative approaches has increased in 
recent years, especially in reading research; neglecting to examine these studies would have resulted in 
the exclusion of a significant number of findings (Shanahan, 2000). Further, including qualitative studies 
in a research synthesis can help explain results by identifying differences in processes used in different 
studies of a program or strategy (Light & Pillemer, 1982). As a counterbalance to the breadth of studies 
included, we rated each study for research quality and described findings in relation to this quality. 

After reading studies and sorting them into the conceptual framework described in the next 
section, we obtained additional research studies from reference citations in the research articles. We 
applied the same inclusion rules to these studies that we used in our initial searches. A total of 1 18 
research studies were synthesized. 



^ Excluded programs are Cooperative Integrated Reading and Composition (CIRC), Direct Instruction (DI) 
Reading, Higher Order Thinking Skills (HOTS), Helping One Student to Succeed (HOSTS), and Reading Recovery. 



ERIC 



12 



18 



Conceptual Framework 

To better analyze and eventually communicate our findings, we read the study abstracts and 
sorted the studies into types of strategies. After trying different sorting frameworks, the following six 
categories of strategies emerged: 

• General Instruction — instructional strategies that are used primarily in whole-group 
settings and that do not fit the definitions of the other five categories 

• Cognitively Oriented Instruction — instruction on strategies for thinking and 
monitoring learning 

• Grouping Structures — strategies for grouping students in a classroom for delivery of 
instruction 

• Tutoring — one-on-one instruction to a student by an adult or older student 

• Peer Tutoring — pairs of students working together in the learning process for 
mutual benefit 

• Computer-Assisted Instruction — use of a computer to present materials for 
instruction and to monitor learning 

A researcher assigned each study to one of the six categories based on the predominant strategy 
that the study addressed. A second researcher verified the category assignment. Where differences in 
assignment occurred, the researchers discussed and resolved their differences based on the definition of 
the category strategy and the strategy’s emphasis in the study. Additional refinement in sorting occurred 
after analysis of the studies began. More detailed definitions of each category strategy are provided in the 
next chapters. 

Conceptually, the six strategies represent two general types: approaches and interventions. 

General Instruction and Cognitively Oriented Instruction are approaches to assisting low-achieving 
students. Grouping Structures, Tutoring, Peer Tutoring, and Computer-Assisted Instruction are 
interventions. The primary difference between these two broad types is that the approaches are more 
general and theoretically based, whereas the interventions are more specific and applied. 

Coding of Studies 

An instrument was developed to code studies for content and quality. The instrument was pilot- 
tested by coding five different studies, three quantitative and two qualitative. The instrument subsequently 
was revised by consensus to enhance clarity and common understanding of terms used for coding and to 
include information needed in order to adequately describe study characteristics and quality. The 
instrument has four major sections: nature of the intervention, research design, data analyses (for 
quantitative studies), and quality of the study. The coding instrument can be found in Appendix A. 

Content coding. Each study was coded for descriptive information about the program or 
intervention strategy being researched. This information included the nature and goals of the strategy, 
how the study identified students as low achieving, the qualifications of those implementing the 
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strategies, how implementers were assigned to the different groups of students in the study, the amount of 
students’ exposure to the strategy during the course of the study, the subject areas that the strategy 
addressed, and the context of the school in which the strategy occurred. The student sample was coded for 
how students were assigned to different groups; the total number of students in the study and in each 
separate group; measures of group attrition; and student characteristics of grade level, gender, and 
ethnicity. 

To code each study’s research design, we identified the predominant research methodology — 
that is, the one on which study conclusions related to the strategy and performance of low-achieving 
students were based. We described quantitative research as either experimental or quasi-experimental 
research. Studies classified as experimental were those in which students were randomly assigned to 
treatment or control/comparison groups. The treatment group referred to those students for whom a 
specific strategy was implemented to raise achievement. The control group referred to students who 
received normal teacher-mediated instruction without the strategy. Comparison groups received 
combinations of strategies or different strategies. Studies classified as quasi-experimental were those that 
did not randomly assign students to control or comparison groups. For these studies, we described 
procedures used to equate or match the different groups in the studies. Most often, teachers, rather than 
students, were assigned to strategies, resulting in very few truly experimental studies. We also coded 
quantitative designs with respect to whether students were pretested on achievement prior to strategy 
implementation and then posttested or were only posttested. 

We described qualitative research designs as case studies, action research studies, studies using 
grounded theory, and ethnographic studies. We noted when qualitative studies used more than one of 
these approaches. For both quantitative and qualitative studies, we described the relevant findings that 
supported use of the strategy as a way to assist low-achieving students. 

Quality coding. In coding the quality of the research studies reviewed, our first consideration was 
whether the study provided sufficient information to index the quality of the research. If sufficient 
information existed, we used Cook and Campbell’s (1979) framework on threats to validity and coded 
quantitative studies for information related to 13 threats. For example, one threat to validity is the loss of 
students from the treatment group prior to the study’s conclusion. If those who drop out have lower 
achievement scores than those who stay, the conclusions could be invalid because results might be due 
either to the strategy or to the type of students remaining in the treatment group. To demonstrate 
consideration of this threat to validity in the findings, a researcher would need to document and discuss 
the number and characteristics of students who left the treatment and control/comparison groups. 

Qualitative studies were coded for information related to 10 threats to validity. This coding 
emphasized the objectivity and reliability of methods used and the degree to which results are credible 
and can be transferred to other settings (Miles & Huberman, 1994). For example, we coded whether 
constructs used for analysis of qualitative data were clearly defined and whether the researcher conducted 
a search for evidence to disconfirm his or her explanations for the findings. 

After coding each study for quality characteristics, we assigned a quality index of low, medium, 
or high. The index was used to communicate an overall rating with respect to the validity of the study’s 
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conclusions. The higher the quality index, the greater the evidence of the study’s validity. We excluded 
studies from the synthesis for which there was insufficient information to assign a quality index. 
Exclusions of this type are discussed separately for each strategy in the next chapters. 

Analyses and Results 

Although each strategy category was assigned to a different researcher for synthesis, researchers 
followed common procedures for identifying subcategories, evaluating and analyzing studies, and 
presenting results. These procedures were jointly developed prior to data analyses and modified through 
frequent discussions as data analyses were completed. We worked closely as a team throughout the 
process and read and critiqued one another’s chapters, developing common approaches to the writing. The 
use of “we” throughout the report reflects this collaborative study approach. 

The decision about whether to conduct a meta-analysis (quantitative synthesis) of the studies in 
each strategy category depended on the following criteria: 

• Whether a sufficient number of studies provided the necessary quantitative 
information needed to compute effect sizes^ 

• Whether a single effect size, the result of a meta-analysis, would provide meaningful 
and useful information within the context of the category 

The chapters that follow describe the analysis and results of a synthesis of research on each of the 
six strategy categories. Each of the chapters, which follow the same general format, includes tables of the 
studies analyzed. Individual studies are elaborated on if they have one or more of the following 
characteristics: high-quality research, informative for practitioners, highlights an implementation issue, is 
one of a combination of studies that give the most complete picture of the strategy. By adapting chapter 
content to the characteristics of studies for each strategy, we were able to more completely discuss results 
than if we had used a uniform presentation. For example, most of the research on computer-assisted 
instruction was quantitative and with sufficient sample sizes to allow for meta-analysis. However, for 
tutoring, the small samples sizes of studies necessitated a different type of analysis and presentation. 

In drawing conclusions about the studies in each chapter, we focused on the extent and quality of 
the research. In some cases, there was insufficient evidence based on the number and quality of the 
studies to determine a result. In other cases, there were some high-quality studies that suggested 
preliminary findings. The final chapter suggests some overall conclusions about the six syntheses in 
relation to the research problem. 



^ Effect size refers to the magnitude of the effect of a strategy/intervention on an outcome such as student 
achievement. In general, the larger the effect size, the stronger the relationship between the strategy/intervention and 
the outcome. Calculation of an effect size requires the following quantitative information: sample size, mean, 
standard deviations for both the strategy group and the control/comparison group, or the value for a / or F statistic 
and the sample size. (For an explanation of the practical use of effect size, consult Marzano, Pickering, and Pollock, 
2001 .) 
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CHAPTER 2 

GENERAL INSTRUCTION 



Sheila A. Arens 

Having limited resources outside the classroom (or lacking such resources altogether), the only 
opportunity that some students have to become literate — to learn to read or compute — > is in the 
classroom. This places an enormous burden on classroom teachers, particularly teachers of students who 
exhibit large variations in academic performance. Although teachers are encountering and struggling with 
competing demands on resources (e.g., time, effort, materials), they must make decisions about their 
practice. Moreover, given the current emphasis on standards-based education aimed at holding all 
students to the same standards of learning and achievement, the expectation that teachers will improve 
student achievement toward standards necessarily influences classroom decisions for low-achieving 
students. 

Clearly, difficulties emerge when teachers consider changing their classroom practices. 

Ostensibly one advantage of variations in general instruction provided within the context of the regular 
classroom is that students labeled “at risk” or “educationally disadvantaged” will be taught the same skills 
as their peers. A primary disadvantage is that approaches in whole-group settings tend to minimize or 
limit teachers’ ability to focus additional energies on their low-achieving students. 

Background 

The approaches to instruction reviewed in the following chapters offer viable strategies for 
teaching children who are low achieving or academically at risk. This chapter provides an overview of 
general instruction that is not as easily categorized by the nature of the intervention. We first categorize 
these approaches conceptually and then provide details about what works for implementation in the 
traditional classroom. The resulting broad conceptual subcategories of behaviorism and constructivism 
overlap with and encompass strategies presented in other chapters. We propose these subcategories in this 
chapter only as a means to better understand the ideology underlying given approaches and to suggest one 
means of understanding practice. Moreover, the overlap of aspects of practices presented in this chapter 
with aspects of strategies discussed in later chapters not only suggests a degree of coherence among 
strategies but also suggests difficulties that arise when attempting to differentiate the impacts of effective 
approaches. Thus, a strategy presented in this chapter may have aspects or specific elements in common 
with a strategy presented in later chapters, despite the fact that the interventions or practices appear to 
differ. When both strategies are deemed efficacious, it becomes difficult to determine whether this is due 
to the elements the strategies share in common or to the combination of convergent and divergent 
elements. 

This chapter addresses the question: What can teachers do in their regular classrooms (in 
primarily whole-group settings) to effect positive gains in achievement? Although a diverse array of 
strategies for regular classrooms might be called “general instruction,” conceptually these approaches to 
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general instruction fall into two general subcategories grounded in theories of knowledge — a 
behaviorist/cognitive perspective, and a constructivist/ developmental/ socio-cultural perspective. 

Methodology 



Selection/Exclusion of Studies 

Based on the criteria described in Chapter One, an initial set of 26 studies was reviewed for 
inclusion. We excluded 1 1 studies from the final set for the following reasons: (1) the intervention used 
was either insufficiently developed or not clear, or multiple strategies were used without differentiation 
among these strategies (n = 6); (2) although the approaches identified were used in classrooms, the 
broader focus of the intervention (systemic) rendered it too complex for classroom implementation (n = 

2); and (3) the research was compromised by design flaws and other biases such that the results could not 
be trusted (n = 3). 

Subcategories within General Instruction 

As previously mentioned, studies in the area of general instruction are conceptually conceived of 
as comprised by two subcategories: behaviorism and constructivism. Of the 15 studies that met inclusion 
criteria, we identified 10 studies as behaviorist and five studies as constructivist. 

One approach to teaching and learning presented in this chapter corresponds to an underlying 
belief that students should be taught discrete skills through linear, somewhat scripted, and explicit 
instruction. Teachers and/or external agents (e.g., materials, computer programs) serve as the purveyors of 
knowledge and act as bearers of the skills and rules necessary for mastery. Researchers and educators 
label this a behaviorist approach (Oakes, 1986). The second approach — a constructivist approach — 
stresses the value of interaction (learners construct knowledge with teachers or peers in the social context) 
and endorses weaving students’ experiences into teaching and learning activities (Oakes, 1986). 

Typically, the importance of “meaning making” is elevated over the importance of learning discrete skills. 
Historically, these approaches have been cast into a dichotomy; however, approaches to teaching and 
learning often are grounded in a more pragmatic tact than a theoretical or philosophical one. 

Consequently, some educators have blended characteristics from these two theories of knowledge. For 
purposes of this research synthesis, studies with this blend are presented as a constructivist approach. The 
remainder of this chapter provides further detail of the conceptual subcategories and the studies included 
in each. 



Behaviorist Approaches 

Given the long history of behaviorist approaches to teaching, it is not surprising to find that a 
large number of studies have been conducted around the efficacy of the many approaches derived from 
this theoretical perspective. Proponents of compensatory education have suggested that learning the 
basics is foundational, that mastering the basics is essential to developing advanced skills. Most advocates 
of behaviorist strategies (at least implicitly) embrace the assumption that knowledge is incremental and 
discrete. Despite the large body of evidence about such programs, we did not include many of these 




20 



25 



studies because they did not examine the impact of the intervention on student achievement, because the 
primary focus was not low-achieving students, or because the interventions were conducted and studied 
outside of the regular classroom. 

One behaviorist strategy captured in this review is explicit instruction, which represents many of 
the strategies housed within a broader behaviorist tradition. These models represent 

a structured approach to teaching which uses scripted formats, extended practice and 
review, a fast pace of instruction, and student responses. . .[it] takes children through 
learning steps systematically, helping them see both the purpose and the result of each 
step. . . . Direct instruction does not mean repetition. It does mean leading students 
through a process and teaching them to use that process as a skill to master other 
academic tasks. (Lewis, Wilson, & McLaughlin, 1992, p. 264) 

Other approaches classified in the behaviorist subcategory include less prescriptive approaches 
with a primary reliance on materials or structured experiences to facilitate learning and approaches 
requiring repeated practice. 

Behaviorist approaches sometimes seem to infuse constructivist ideas — for instance, students 
may be placed in dyads or small groups — however, for the most part the components of explicit 
instruction associated with this approach prevail: independent practice, memorization of vocabulary or 
computations, use of worksheets and guided practice, and a perspective of teacher as bearer of knowledge 
and skills. 

For instance, phonics-based instruction tends to stress learning in teacher-directed incremental 
steps that are relatively scripted; typically these phonics-based approaches are behaviorist in nature. In a 
review of research, Ebaugh (2000) suggests that the past 30 years have produced compelling evidence 
that phonological skills are a strong predictor of the facility with which students learn to read and that 
early intervention (prior to third grade) in remedial skills is crucial to effecting positive results. The 
phonological skills considered most critical for students to grasp are awareness of sounds and letters, 
memory, and decoding. Other researchers have offered similar prognoses for children who fail to reach 
basic language proficiency in their early development. For instance, Clapp (cited in DiChiara, 2001) 
“found that children who fail to develop proficiency in language during the first years of life are up to six 
times more likely to experience reading problems when they go to school” (p. 2). However, based on the 
research reviewed herein, we are unable to make strong claims related to the overall efficacy of such 
incremental, guided instruction representative of the behaviorist approach. 

Overview of Studies 

The majority of studies reviewed in the behaviorist subcategory examined impacts on elementary 
students (n = 7); one study focused on students at the middle school level; participants in the remaining 
two studies were in high school (see Table 2.1). In terms of subject matter, all but one of the studies 
focused — at least in part — on reading; one study examined mathematics. Finally, low-achieving 
children were defined in various ways — some based on test scores and others based on socioeconomic 
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background or other predictive indicators. In the next section, details on studies that were coded as high 
quality or were particularly noteworthy are provided. 



Table 2.1. Characteristics of Studies on Behaviorist General Instruction 



Author(s) & Year 


Treat 

ment 

size 


Grade 


Student 

Description 


Strategy 

Description^ 


Subject 

Matter 


Type of 
comparison 


Results^ 


Foorman, Francis, 
Fletcher, Mehta, & 
Schatschneider 
(1998) 


285 


1-2 


test scores; 
SES 


explicit 

instruction 


reading 


3 gps pre-post 
nonrandom 


Mostly 

positive 


George-Remy 

(1991) 


7 


K 


test scores 


explicit 
instruction; 
repeated practice 


reading 


2 gps pre-post 
nonrandom 


All 

positive 


Goldenberg (1994) 


24 


K 


SES 


directed 

materials; 

additional 

exposure 


reading 


3 gps post 
nonrandom 


Even 


Gomer (1994) 


65 


9 


SES; test 
scores 


explicit 

instruction; 

additional 

exposure 


reading 


1 gp pre-post 
nonrandom 


All 

positive 


Hennenfent & 
Russell (2001) 


27 


K, 5 


SES, test 
scores 


explicit 

instruction 


reading, 

writing 


1 gp post 
nonrandom; 1 gp 
post nonrandom 


Mostly 

positive 


Jenkins, Jewell, 
Leicester, Jenkins, 
& Troutner (1991) 


42 


1-6 


mild 

disabilities, 

LD 


explicit 
instruction; in- 
class assistance 


math, 

reading, 

writing 


2 gps pre post 
nonrandom 


Even 


Marseglia (1997) 


20 


1 


test scores 


repeated practice 


reading 


2 gps pre post 
nonrandom 


Even 


Morris (1998) 


67 


9 


test scores 


explicit 

instruction; 

integrated 

curriculum 


math, 

reading, 

writing, 

science, 

health 


1 gp pre post 
nonrandom 


Even 


Su(1990) 


104 


4 


test scores 


repeated practice 


math 


1 gp pre post 
nonrandom 


All 

positive 


Turner (1993) 


54 


6-8 


test scores 


explicit 

instruction; 

additional 

exposure 


reading 


1 gp pre post 
nonrandom 


Even 



“The type of instruction. 

’’The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Findings 

Behaviorist Approach: Direct Instruction. Seven studies included in this review explicitly 
examined the effects of interventions categorized as direct instruction; however, arguably, all of the 
studies in the behaviorist subcategory have components of such an approach. These studies examined the 
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impact of explicit instruction in such areas as decoding, that is, determining the constitutive elements of 
text and locating elements that facilitate interpretation or inference. 

The highest quality study of direct instruction behaviorist approaches among the research 
reviewed was an exploration of differences among a variety of approaches to reading instruction. 
Foorman, Francis, Fletcher, Mehta, and Schatschneider (1998) compared the effects of three different 
decoding strategies to a control group that received the standard district-endorsed instruction. The 
experimental groups consisted of (1) highly explicit decoding in which phonemic rules were practiced 
with controlled vocabulary text; (2) an embedded phonics approach in which letter sound and spelling 
patterns instruction was provided within the context of selected, predictable texts; and (3) implicit 
decoding, in which feedback on reading literature was provided and meaning was emphasized. Foorman 
et al. concluded that “instructional intervention makes a difference [but]. . . not all instructional 
approaches have the same impact” (p. 51). That is, students who were directly instructed in alphabet 
principles showed more rapid rate of word-reading skill than those children who received indirect 
instruction. Coupled with a belief that students must learn vocabulary and decoding early in their 
education and that these skills are fundamental, these findings may suggest a viable means of rapidly 
training primary school students in the basics of reading. 

Other studies included behaviorist interventions (some of which were direct instruction 
approaches) coupled with additional materials or out-of-school experiences, such as library visits, parental 
support/at-home guidance. For instance, Goldenberg (1994) reports findings from a follow-up study that 
examined the impact of general instruction (with little exposure to written texts), highly structured, 
academic instruction (direct instruction in letters, sounds, words; drills, practice and seat work), or 
instruction supplemented with predictable Spanish books (used at school and possibly at home) to 
Spanish-speaking beginning readers. Findings suggest that providing Spanish books resulted in 
measurable gains in literacy development compared to general instruction (with reported effect sizes of .7 
to .8). However, students receiving structured, direct instruction showed even greater gains (with reported 
effect sizes of 1 to 3). 

Another approach that emphasizes the role of external agents (e.g., teacher as guide) has resulted 
from the decline in use of pull-out programs. Although primarily used with mainstreamed special 
education students, the use of in-class consultants has been identified as a viable option for teachers of 
otherwise low-achieving students (Lewis et al., 1992). One study in the set reviewed herein (Jenkins et 
al., 1991) utilized a classroom consultant to facilitate improvement in skill areas among at-risk students. 
Unfortunately, it was not possible to determine what the intervention actually was in terms of how in- 
class consultants interacted with the students. 

Of the seven research studies reviewed for this chapter that examined direct instruction methods, 
only one was rated high quality. Indeed, six failed to adequately control for alternative explanations, 
particularly gains as a result of normal growth. Consequently, reviewing these approaches must be 
accompanied by caution regarding the supporting evidence. Several issues emerged with the research 
studies reporting mixed results regarding efficacy focused on learning the rules through direct instruction 
in skill areas (e.g., letter-sound associations, decoding strategies, basic phonics principles, word family 
concepts, punctuation skills, reading skills in isolation and in context, applying text elements, learning 
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and applying high frequency words, learning and applying vocabulary skills, learning and applying 
phonemic rules, and learning and applying figurative language). 

Behaviorist Approach: Repeated Instruction, Another intervention showing moderate effects 
was repeated practice — having students practice and receive feedback in both reading and mathematics 
until mastery is achieved. The assumption with these approaches is that repeated exposure (sometimes 
coupled with explicit instructions) will increase skill levels or comprehension (George-Remy, 1991). 
Obviously, repeated-practice strategies can be creatively approached; some interventions utilized within- 
and between-class competitions in conjunction with flashcards and structured recall episodes (Su, 1990); 
others used audio-taped student storytelling in conjunction with listening to stories, reading aloud with a 
peer partner, or round robin reading (Gomer, 1994; Marseglia, 1997). However, given the number of 
design flaws associated with the research reviewed on these interventions, additional research is certainly 
warranted. These flaws included failing to sufficiently report data, failing to take into account pre-test 
scores or account for subject maturation, or drawing unsupported inferences about the treatment 
(mismatch between evidence and claims). 

Constructivist Approaches 

Proponents of the constructivist view of teaching and learning believe that “concepts are not 
something given to the child but rather are constructed by the child who interacts with that environment” 
(Means & Knapp, 1991, p. 9). Moreover, they resist embracing a deficit view of low-achieving students 
that 



focuses on the inability of at-risk students to complete tasks and encourages decomposing 
complex tasks into . . . levels of ability that are structured linearly from lower-level . . . 
skills like spelling and punctuation to higher level . . . intellectual processes like synthesis 
or critical analysis. (Bryson & Scardamalia, 1991, p. 155) 

As opposed to using closed-ended questions about known information and recounts of text or 
formulae such as round robin reading, orientation questions focused on vocabulary, guidance in decoding, 
and rote memorization, interventions falling into this subcategory encourage students to engage with texts 
and to attempt to understand at a deeper level. 

Overview of Studies 

All five studies reviewed in the constructivist subcategory examined impacts on elementary 
students (see Table 2.2). In terms of subject matter, all of the studies collected data on reading or writing; 
one study examined mathematics achievement as well. As in the studies of behaviorist approaches, at-risk 
children were defined in these studies in a variety of ways. In the next section, details on studies that were 
coded higher quality or were particularly noteworthy are provided. 
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Table 2.2. Characteristics of Studies on General Constructivist Instruction 



Author(s) & Year 


Treat 

ment 

size 


Grade 


Student 

Description 


Strategy 

Description^ 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Brown, Pressley, 
Van Meter, & 
Schuder(1996) 


30 


2 


test scores; 
grades; 
teacher 
assessment 


meaning-based 

instruction 


reading 


2 gps pre post 
nonrandom 


Mostly 

positive 


Knapp, Shields, & 
Turnbull (1992) 


NA 

(140 

classr 

ooms) 


1-6 


test scores 


meaning-based 

instruction 


math, 

reading, 

writing 


2 gps pre post 
nonrandom 


Mostly 

positive 


Simmons, Fuchs, 
Fuchs, Mathes, & 
Hodges (1995) 


29 


2-5 


test scores 


balanced 


reading 


2 gps pre post 
nonrandom 


Even 


Sylva (2000) 


105 


2 


LD or 

teacher 

assessment 


balanced 


writing 


2 gps pre post 
nonrandom 


Even 


Wolf(1998) 


17 


3^ 


school 
identified 
for pull out 
or remedial 


interactive 


reading 


qualitative 


Even 



“The type of instruction. 

*^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Findings 

In general, the research in this area examined attempts to emphasize meaning and eschew 
behaviorist methods (such as direct instruction) or attempts to infuse some behaviorist strategies within a 
predominantly constructivist approach. We rated all of the studies included in this subcategory as high or 
mid to high quality. 

The most compelling evidence for constructivist approaches to teaching is offered by a study 
coded as high quality in which researchers engaged in a three-year study of 140 classrooms in 15 schools 
(Knapp et al., 1992). Researchers categorized teachers’ existing practices as either traditional 
(behaviorist) or alternative (constructivist). Knapp and his associates found that, regardless of subject 
area, instruction that emphasized meaning and understanding led to greater gains in advanced skills (e.g., 
mathematical understanding, problem-solving ability, reading comprehension, written expression). 
Moreover, findings suggested that an emphasis on meaning worked “at least as well for low-performing 
as high-performing students” (p. 27). The researchers viewed this as evidence counter to the notion that 
skills building is foundational to advanced understanding. Moreover, the researchers found that the 
amount of instructional time, the time spent teaching discrete skills, and individual characteristics of 
teachers did not independently account for the findings. However, they did find that outcome scores 
related to student learning were impacted by poverty-related factors such as support at home and 
understanding of school culture. 
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Wolf (1998) worries that packaged “skill and drill” programs, although perhaps useful for 
inculcating socially acceptable classroom behaviors, fail to engage students in deeper understanding. She 
suggests that when the act of reading is viewed simply in terms of comprehension of text, the implicit 
assumption is always that the meaning inheres in the text itself and that there are no other interpretations 
of author intent, author meaning, or ways that one could connect the text to one’s own experiences. Wolf 
reported on classroom theatre as an alternative approach to teaching literacy in a qualitative study coded 
as having mid-level quality. She examined impacts in terms of enhancing low-achieving students’ reading 
and comprehension skills by building connections through text interpretations. Although Wolfs results 
were mixed with respect to longer term benefits of the approach, she continues to argue for the use of an 
alternative method of teaching literacy, while conceding that such experiences need to be accompanied by 
additional structure. She suggests that those skills formerly considered prerequisites (e.g., structural 
elements, grammar, phonics) may not develop naturally in authentic (real-world) contexts, thus some 
blend with behaviorist strategies may be most beneficial. 

In a study rated mid to high quality that compared a blended approach to instruction (explicit 
instruction combined with a reflective dialogue journal) with a wholly constructivist approach (reflective 
journal) among students in the primary grades, Sylva (2000) found no significant differences over time in 
achievement scores. However, the researcher also noted that problems with treatment fidelity likely 
impacted findings. That is, Sylva suggested that teachers may not have maintained a completely blended 
approach or a completely constructivist approach. Other examinations of the impact of direct or explicit 
teaching provided some evidence against strictly behaviorist approaches. One such study rated as high 
quality indicated “no reliable independent effect” for explicit teaching (Simmons et al., 1995, p. 401) 
when students receiving explicit teaching were compared with students either receiving a combination of 
behaviorist and constructivist approaches (blending explicit teaching and peer tutoring) or students in a 
control group. The researchers found that breaking student tasks into small steps with repeated practice 
was no more effective for low-achieving students than other approaches. It is important to note, however, 
that Simmons and colleagues were unable to ensure the fidelity of teaching approaches; that is, some 
teachers who were trained to teach explicitly failed to teach more explicitly than control group teachers. 

On the other hand, other research studies have provided evidence that blended approaches — 
perhaps more patchwork-like strategies in which inclusion of constructivist modalities is inconsistent — 
may not be as effective as wholly constructivist approaches. Brown et al. (1996) compared the effect of 
“transactional teaching” (linking reading to prior knowledge, reading in groups toward co-construction of 
knowledge, and group engagement to co-determine one another’s meanings about texts) with 
conventional teaching (represented by an amalgam of behaviorist and constructivist approaches). The 
researchers of this study rated as high quality found positive short- and long-term impacts in terms of 
student acquisition of information, understanding and personalization of material, awareness of strategies, 
and reading comprehension among teachers using the transactional teaching approach. 

Across the five constructivist studies reviewed for this chapter, the combined findings of two of 
the studies found that low-achieving students benefited as much as their higher achieving counterparts by 
instruction that emphasized meaning construction and that repeated practice and learning in steps did not 
provide an additional advantage. However, blended approaches that used both behaviorist-like strategies 
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and constructivist-like strategies for instruction also were shown to work with at-risk students. Future 
research efforts that not only compare the efficacy of wholly behaviorist approaches, wholly 
constructivist approaches, and a blend of these, but also control for factors such as degree of teacher 
adherence to the approach, are needed. 

Summary of Findings Across Studies of General Instruction 

It is obvious from this synthesis of research categorized as General Instruction that little evidence 
exists to demonstrate the efficacy of different approaches under varied contexts and characteristics of 
students. Given our focus, unfortunately we do not have a large enough body of evidence to suggest 
whether behaviorist strategies are superior to constructivist strategies or vice versa. Nor do we have 
sufficient data to support claims about the efficacy of a blended approach. It is important to note again 
that behaviorist approaches to instruction may have merit, but only a small number of studies met the 
inclusion criteria for this chapter of the review. Of those studies falling in the behaviorist subcategory, 
only one was coded as high quality and two as mid-level quality. On the other hand, all five of the studies 
comprising the constructivist subcategory were coded as either high or mid to high quality. Claims or 
recommendations for or against behaviorist or constructivist subcategories are clearly not warranted given 
the body of evidence reviewed. Much has been said about both of these conceptual understandings of 
teaching and much remains to be discovered. Comparative studies that carefully examine these two 
approaches should be consulted. 

On a socio-political level, arguments against behaviorist approaches suggest that teachers 
embracing a deficit perspective of low-achieving students are more directive, teach discrete and isolated 
facts, and offer few opportunities for exercising higher order thinking. For constructivists, this view of 
learners further disadvantages low-achieving students — their peers progress, but they are held back until 
they master the basics. Thus, the gap between high- and low-achieving students widens. At-risk students 
may begin to equate rote skills and rules with learning rather than viewing learning as a process that 
includes higher order thinking skills and active engagement (Palinscar & Klenk, 1991). Additional 
attention to the differential impacts of varied instructional approaches for diverse students may provide 
necessary information about the intended and unintended consequences of adopting such approaches. 

Naturally, proponents of behaviorist approaches have strongly attacked constructivists for failing 
to meet the needs of all children in neglecting the basics. Other counter-arguments to constructivist 
approaches suggest that failing to provide at-risk students with instructional leaders places them at a 
further disadvantage because their access to teachers as sources of knowledge is effectively cut off 
(Palinscar & Klenk, 1991). 



A Final Word to Practitioners 

Behaviorist approaches to instruction are typically characterized by instructors delivering 
information to students, students engaging in independent learning, and students practicing and receiving 
feedback regarding their acquisition of the target set of skills. Tasks are typically broken down into 
smaller units to facilitate learning — though expository teaching (providing explanations of facts and 
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ideas) also may be present (Woolfolk, 1998). On the other hand, constructivist approaches to instruction 
consist of instructors facilitating the discovery of information, learning situations that are authentic, and 
students engaged in collaborative learning activities aimed at the co-construction of meaning. Students are 
encouraged to be active discoverers and cooperative learners, and the relationship between teaching and 
learning is viewed as reciprocal and collaborative (Woolfolk, 1998). 

Philosophically, the schism between these approaches to instruction runs deep. Although the 
pendulum continues to swing between strong behaviorist and strong constructivist approaches to teaching 
and learning, some have opted to take a middle ground and borrow components or aspects of what seems 
to work from both sides. For instance, Danoff, Harris, and Graham (1993) suggest that a body of research 
“demonstrates that many students with learning problems do not acquire a variety of cognitive and 
metacognitive strategies unless detailed and explicit instruction is provided” (p. 297). Likewise, Resnick, 
Bill, Lesgold, and Leer (1991) maintain that approaches embracing “more careful explanations, more 
practice, and more frequent testing to monitor progress . . . seem to work — up to a point” (p. 27). The 
incorporation of direct strategy instruction within classrooms that also honors students’ context may be 
what works. This balanced approach would carefully consider what should be taught explicitly versus 
whether to teach explicitly (McIntyre, 1995), yet also consider how to capitalize on life experiences and 
discovery. It is to these questions and concerns that we will be continually drawn. 

Readers interested in learning more about behaviorist approaches to instruction may wish to 
consult Bransford, Brown, and Cocking (2000) or Woolfolk (1998). Each of these study provides a broad 
overview of theories of learning, their history, and their application in the classroom, as well as additional 
details about specific approaches and references to foundational works. Readers seeking information on 
Direct Instruction, a behaviorist program outside the scope of this synthesis (see Chapter 1), may find the 
following resources useful: Adams and Engelmann (1996); Becker and Camine (1980); Engelmann and 
Camine (1991); or Gersten, Woodward, and Darch (1986). Finally, readers interested in constructivist 
approaches to instruction may wish to consult Oakes and Lipton (1999) or Resnick, Bill, Lesgold, and 
Leer (1991). Oakes and Lipton provide a comprehensive overview of the constructivist theory of learning 
and embed it in classroom practice. Resnick and her colleagues provide an historical overview of 
alternative classroom practices juxtaposed with traditional practices as well as a fairly balanced critique of 
both perspectives. 



References 

(*=Studies included in the synthesis) 

Adams, G. L., & Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond 
Seattle, WA: Educational Achievement Systems. 

Becker, W. C., & Camine, D. W. (1980). Direct Instmction: An effective approach to educational 
intervention with disadvantaged and low performers. In B. B. Lahey & A. E. Kadzin (Eds.), 
Advances in clinical child psychology (vol. 3) (pp. 429-473). New York: Plenum. 

Bransford, J. D., Brown, A. L., & Cocking, R. (Eds.). (2000). How people learn: Brain, mind, experience 
and school. Washington, DC: National Academy Press. 



28 33 



♦Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental validation of 
transactional strategies instruction with low-achieving second-grade readers. Journal of 
Educational Psychology, 88{\), 18-37. 

Bryson M., & Scardamalia, M. (1991). Teaching writing to students at risk for academic failure. In B. 

Means, C. Chelemer, & M. S. Knapp (Eds.), Teaching advanced skills to at-risk students: Views 
from research and practice (pp. 141—167). San Francisco: Jossey-Bass. 

Cooper, H., Charlton, K., Valentine, J. C., & Muhlenbruck, L. (2000). Making the most of summer 

school: A meta-analytic and narrative review. Monographs of the Society for Research in Child 
Development, Serial No. 260, 65(1). 

Danoff, B., Harris, K. R., & Graham, S. (1993). Incorporating strategy instruction within the writing 
process in the regular classroom: Effects on the writing of students with and without learning 
disabilities. Journal of Reading Behavior, 25(3), 295-322. 

DiChiara, L. E. (2001). The effectiveness of direct instruction versus traditional basal reading instruction 
as it pertains to at-risk youth. Dissertation Abstracts International, 67(07), 2574. (UMI No. 
9978316) 

Ebaugh, J. C. (2000). The effects of fluency instruction on the literacy development of at-risk first 
graders. Dissertation Abstracts International, 67(06), 2238. (UMI No. 9975346) 

Engelmann, S., & Camine, D. (1991). Theory of instruction: Principles and applications. Eugene, OR: 
ADI Press. 

♦Foorman, B. R., Francis, D. J., Fletcher, J. M., Mehta, P., & Schatschneider, C. (1998). The role of 
instruction in learning to read: Preventing reading failure in at-risk children. Journal of 
Educational Psychology, P0(1), 37-55. 

*George-Remy, C. Y. (1991). Improvement in kindergarten children’s expressions with the language arts: 
Short-term instruction on the parts of a story and story re-telling with teacher feedback. Early 
Child Development & Care, 73^ 19-27. 

Gersten, R., Woodward, J., & Darch, C. (1986). Direct Instruction: A research-based approach to 
curriculum design and teaching. Exceptional Children, 55, 17-31. 

♦Goldenberg, C. (1994). Promoting early literacy development among Spanish-speaking children: 

Lessons from two studies. In E. H. Hiebert & B. M. Taylor (Eds.), Getting reading right from the 
start (pp. 17 1-1 99). Boston: Allyn & Bacon. 

♦Gorner, J. R. (1994). Improving the reading skills of ninth grade underachievers by instilling a desire to 
read. Unpublished doctoral dissertation. Nova University, Florida. 

♦Hennenfent, M., & Russell, J. (2001). Increasing independent reading levels using an integrated 
approach emphasizing direct reading instruction. Unpublished master’s thesis, St. Xavier 
University, Chicago, Illinois. 

♦Jenkins, J. R., Jewell, M., Leicester, N., Jenkins, L., & Troutner, N. M. (1991). Development of a school 
building model for educating students with handicaps an at-risk students in general education 
classrooms. Journal of Learning Disabilities, 24{5\ 311-320. 



ERJC 



29 34 



*Knapp, M. S., Shields, P. M., & Turnbull, B. J. (1992). Academic challenge for the children of poverty: 
Study of academic instruction for disadvantaged children. Summary report. Washington, DC: 
U.S. Department of Education, Office of Policy and Planning. 

Lewis, A., Wilson S. M., & McLaughlin, T. F. (1992). Resource room and the consulting model for 
servicing low achieving students: A review and analysis. B.C. Journal of Special Education, 
7d(3), 259-281. 

*Marseglia, P. (1997). The effect of repeated readings on the fluency of high and low level ability readers 
in a first grade class. Unpublished master’s thesis, Kean College of New Jersey. 

Means, B., & Knapp, M. S. (1991). Introduction: Rethinking teaching for disadvantaged students. In B. 
Means, C. Chelemer, & M. S. Knapp (Eds.), Teaching advanced skills to at-risk students: Views 
from research and practice (pp. 1-26). San Francisco: Jossey-Bass. 

McIntyre, E. (1995). Teaching and learning writing skills in a low-SES, urban primary classroom. 

Journal of Reading Behavior, 27(2), 213-242. 

*Morris, L. L. (1998). The effects of integrated curriculum on 9th grade at-risk students. Unpublished 
master’s thesis, St. Xavier University, Chicago, Illinois. 

Oakes, J. (1986). Tracking, inequality, and the rhetoric of reform: Why schools don’t change. Journal of 
Education, 168{\\ 60-80. 

Oakes, J., & Lipton, M. (1999). Teaching to change the world. Boston: McGraw-Hill. 

Palinscar, A. S., & Klenk, L. H. (1991). Dialogues promoting reading comprehension. In B. Means, C. 
Chelemer, & M. S. Knapp (Eds.), Teaching advanced skills to at-risk students: Views from 
research and practice (pp. 112—130). San Francisco: Jossey-Bass. 

Resnick, L. B., Bill, V. L, Lesgold, S. B., & Leer, M. N. (1991). Thinking in arithmetic class. In B. 

Means, C. Chelemer, & M. S. Knapp (Eds.), Teaching advanced skills to at-risk students: Views 
from research and practice (pp. 27-53). San Francisco: Jossey-Bass. 

*Simmons, D. C., Fuchs, L. S., Fuchs, D., Mathes, P., 8c Hodges, J. P. (1995). Effects of explicit teaching 
and peer tutoring on the reading achievement of learning-disabled and low-performing students in 
regular classrooms. The Elementary School Journal, P5(5), 387-408. 

*Su, H. F H. (1990). Increasing fourth grade math achievement with improved instructional strategies. 
Unpublished doctoral dissertation. Nova University, Florida. 

*Sylva, J. A. (2000). Teaching literacy skills to students at risk: A writing intervention. Dissertation 
Abstracts International, 60{\2-A), 4323. (UMI No. 9954165) 

^Turner, T. N. (1993). Improving reading comprehension achievement of sixth, seventh, and eighth grade 
underachievers. Unpublished doctoral dissertation. Nova University, Florida. 

*Wolf, S. A. (1998). The flight of reading: Shifts in instruction, orchestration, and attitudes through 
classroom theatre. Reading Research Quarterly, ii(4), 382-415. 

Woolfolk, A. E. (1998). Educational psychology (7^*^ ed.). Boston: Allyn & Bacon. 



ERJC 



30 



35 



Annotated Bibliography of Selected References 



This annotated bibliography provides additional information on studies included in the research 
synthesis that were identified as being of mid to high quality. 
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subject area, instruction emphasizing meaning and understanding leads to greater gains in advanced skills 
(e.g., mathematical understanding, problem-solving ability, reading comprehension, written expression). 
Moreover, their findings suggested that an emphasis on meaning works “at least as well for low- 
performing as high-performing students” (p. 27). The researchers viewed this as evidence opposing the 
notion that building skills is foundational to advanced understanding. And although the researchers found 
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that the amount of instructional time, the time spent teaching discrete skills and the individual 
characteristics of teachers did not independently account for findings, they did find that learning 
outcomes are impacted by poverty-related factors (e.g., support at home and knowledge of school 
culture). 

Wolf, S. A. (1998). The flight of reading: Shifts in instruction, orchestration, and attitudes 
through classroom theatre. Reading Research Quarterly, 55(4), 382-415. 

In this case study of a classroom that elected to approach teaching literacy through interactive 
classroom theatre as opposed to round-robin reading, Wolf examined impacts on student reading and 
comprehension skills. Classroom theatre is presented as a means of allowing children to build their own 
connections with texts and co-construct interpretations of text. Although changes were documented 
related to improved opinions of reading, increased reading, and successful reading, Wolfs results were 
mixed with respect to longer term benefits of the approach. The researcher acknowledges that alternative 
methods of teaching literacy (such as classroom theatre) are not a panacea for complex, orchestrated 
endeavors such as learning to read but may be best used in conjunction with additional guidance related to 
learning structural elements, grammar, and phonics. 

Brown, R., Pressley, M., Van Meter, P., & Schuder, T. (1996). A quasi-experimental validation of 
transactional strategies instruction with low-achieving second-grade readers. Journal of Educational 
Psychology, 55(1), 18-37. 

In a quasi-experimental study, Brown, Pressley, Van Meter, and Schuder (1996) compared the 
effect of “transactional teaching” (linking reading to prior knowledge, reading in groups toward co- 
construction of knowledge, and group engagement to co-determine one another’s meanings about texts) 
with conventional teaching (represented by an amalgam of behaviorist and constructivist approaches). 

The researchers found positive short- and long-term impacts in terms of student acquisition of 
information, understanding and personalization of material, awareness of strategies, and reading 
comprehension for teachers using the transactional teaching approach. Indeed, Brown and colleagues 
suggest that the second-grade students in the transactional reading group lessons learned more daily 
compared to their conventional counterparts and showed better performance on standardized tests over 
the course of the year. 
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CHAPTER. 3 



COGNITIVELY ORIENTED INSTRUCTION 

Helen S. Apthoip 

Critics of compensatory education highlight the absence of opportunities for students to develop 
higher order problem-solving and reading comprehension abilities (Means & Knapp, 1991a). Observers 
of classroom instruction under this model report unnecessary repetition of content through drill and 
practice, an overemphasis of teacher-directed, whole-group instruction, and few questions and little 
discussion about complex issues. To some observers, these classroom practices have resulted in student 
passivity (Waxman, Padron, & Arnold, 2001). Cognitively oriented instruction, on the other hand, 
emphasizes learning as an active process, assumes that students need to apply cognitive strategies in order 
to learn, and views teaching as a means of facilitating active student mental processing (Waxman et al., 
2001). This chapter focuses on cognitively oriented approaches to instruction as a means for raising 
achievement of low-achieving or at-risk students. 

Background 

In this chapter, cognitively oriented instruction is defined broadly to include instruction designed 
to help students improve the quality of their thinking, become independent learners, and develop 
proficiency at accomplishing complex, higher order, academic tasks (Andrews, Short, & Mulcahy, 1991; 
Jones, 1986; Pressley, Woloshyn, & Associates, 1995). These goals are congruent with the goals of 
standards-based education. In all content areas, standards specify deep understanding of important 
concepts and principles and disciplined, expert thinking as learning targets. All children deserve an 
education that improves the quality of their thinking and prepares them to be lifelong learners. 

Cognitively oriented instruction has the potential to inspire students’ initiation and use of 
powerful strategies, such as recognition of task demands, planning ahead, visual imagery, verbal 
elaboration, systematic problem solving, and self-checking. Cognitively oriented instruction refers to the 
same approach Waxman et al. (2001) call “cognitively guided instruction.” In this chapter, we use the 
term “cognitively oriented instruction” because “cognitively guided instruction” is the term usually 
associated with Cognitively Guided Instruction (GCI), an approach specific to mathematics instruction 
(Peterson, Fennema, & Carpenter, 1991). The research cited by Waxman et al. (2001) supporting the 
effectiveness of this approach primarily involved English Language Learners (e.g., Padron & Knight, 
1989). The present review extends Waxman’ s et al. (2001) review by including studies on the 
effectiveness of cognitively oriented instruction for low-achieving or at-risk students other than English 
Language Learners. 

Cognitively oriented instruction includes both metacognitive and cognitive strategies. These 
strategies serve three common functions across different content areas: (1) planning/preparation and idea 
generation, (2) guidance on “how-to” accomplish tasks, and (3) self-regulation. In the literature, strategies 
serving the first and third functions are described as metacognitive, that is, they involve thinking about 
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one’s own thinking and task demands. Strategies serving the second function are described as cognitive, 
that is, they are the mental processes themselves. The distinction between metacognitive and cognitive 
strategies, however, is often vague and changing. Strategies for reading comprehension, for example, 
once considered cognitive (e.g., establishing a purpose for reading), are now considered metacognitive 
(Dickson, Collins, Simmons, & Kameenui, 1998). 

Cognitively oriented instruction also has four instructional components: (1) presentation of 
strategies (through modeling, explicit teaching, or self-reflection and self-report); (2) exercise of 
strategies (e.g., through guided and independent practice); (3) strategy evaluation, reflection, and 
feedback (provided through instructional conversations and/or cooperative work groups); and (4) the 
social and emotional qualities of the strategies and learning tasks. Cognitively oriented instruction 
“encourages teachers to focus on affective, motivational, metacognitive, developmental, and social factors 
that influence students because they all occur simultaneously and are all critical to students’ learning” 
(Waxman et al., 2001, p. 142). 

In this chapter, we first present our methodology for identifying and including/excluding studies 
in this review. Next we provide an overview of the included studies, describing their study characteristics 
and results. More detailed review of the studies, their findings, and the implications for classroom practice 
are provided under three subcategory headings: Cognitively oriented instruction in (1) Reading, (2) 
Writing and Oral Language, and (3) Mathematics. In each subcategory, both the content of the strategy 
instruction and the methods or components of instruction included in the interventions are described and a 
review of the evidence on their effectiveness is provided. 



Selection/Exclusion of Studies 

This chapter synthesizes research evidence on cognitively oriented instruction, including ideas 
from theoretical articles that explain and define the construct of cognitively oriented instruction, evidence 
from prior research reviews, and evidence from primary studies. In all, four reviews and 15 primary 
studies were included. Of the four reviews, one addressed reading comprehension in particular (Dickson 
et al., 1998), and the other three addressed cognitively oriented instruction across the content areas 
(Means, Chelemer, & Knapp, 1991; Pressley et al., 1995; Waxman et al., 2001). 

To be included in the present review, primary studies had to meet the criteria identified in 
Chapter 1. An initial set of 43 studies was reviewed for inclusion; of these, 15 are included in this chapter. 
Of the 28 studies excluded from the final set, 14 were excluded for failing to meet the criteria identified in 
Chapter 1. Fourteen other studies were excluded because the instruction being studied was not cognitively 
oriented instruction. Instead, these excluded studies examined instruction that was (1) characterized as 
whole language; (2) designed primarily to utilize multimedia, background music, and/or manipulatives; or 
(3) designed to tailor instruction to different learning styles. 
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Overview of Primary Studies 

Table 3. 1 summarizes the characteristics of the set of 15 primary studies. 



Table 3.1. Studies on Cognitively Oriented Instruction for Low-Achieving Students 



Author(s) &Year 


Treat- 

ment 

Size 


Grade 


Student 

Description^ 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Bottge (1999) 


17 


8 


Low- 

achievers 


Mathematics 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Brown (1995) 


55 


8 


At-risk 


Content-area 

reading 


1 grp pre-post 
nonrandom 


Mostly 

positive 


Brown, Pressley, Van 
Meter, & Schuder 
(1995) 


30 


2 


Low- 

achieving 


Reading 


2 grps pre-post 
nonrandom 


All positive 


Cardelle-Elawar (1990) 


80 


6 


Low- 

achieving 


Mathematics 


8 grps pre-post 
random 


Mostly 

positive 


Dermody (1988) 


14 


4 


Low- 

achieving 


Reading 


6 grps pre-post 
random 


Mostly 

positive 


Dimino, Gersten, 
Camine, & Blake 
(1990) 


32 


9 


Low- 

achieving 


Reading 


2 grps pre-post 
random 


All positive 


Englert, Raphael, 
Anderson, Anthony, & 
Stevens (1991) 


52 


4& 5 


Low- 

achievers 


Writing 


3 grps pre-post 
nonrandom 


Mostly 

positive 


Guastello, Beasley, & 
Sinatra (2000) 


62 


7 


Low- 

achieving 


Content-area 

reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Jakupcak, Rushton, 
Jakupcak, & Lundt 
(1996) 


107 


High 

school 


LD and low- 
achievers 


Content-area 

reading 


1 grp pre-post 
nonrandom 


Mostly 

positive 


Ketter & Pool (2001) 


23 


9 


Low- 

achieving 


Writing 


1 grp pre-post 
nonrandom 


Mostly 

positive 


McLain (1991) 


108 


3 & 5 


At-risk 


Reading 


3 grps pre-post 
nonrandom 


Even 


O’Malley, Chamot, 
Stewner-Manzanares, 
Russo, & Kupper 
(1985) 


53 


High 

school 


ELL 


Oral language 


3 grps pre-post 
random 


All positive 


Spiegel, Jackson, 
Graham, & Ware 
(1990) 


17 


6 


At-risk 


Reading 


2 grps pre-post 
random 


Mostly 

negative 


Waxman, deFelix, 
Martinez, Martinez, 
Kjiight & Padron 
(1994) 


52 


1 to 5 


ELL 


Reading 


3 grps pre-post 
nonrandom 


Mostly 

negative 


Wong, Butler, Ficzere, 
& Kuperis (1996) 


18 


8& 9 


LD and low- 
achieving 


Writing 


2 grps pre-post 
nonrandom 


Mostly 

positive 



^Low-achieving refers to students performing below grade level or below average on achievement tests or who met 
eligibility criteria for remedial classes or Title I. refers to students in urban or rural schools and identified in 

the report as at-risk. LD refers to students receiving special education support for identified learning disabilities. 
ELL refers to students learning English as a second language. 

^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 
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As can be noted, nearly all the studies were published in the 1990s. The size of the treatment 
group in seven studies included between 30 and 62 students; the other treatment group sizes ranged above 
and below this, from 9 to 107 students. Five studies were conducted at the elementary level (grades 1-5); 
ten studies, at the secondary level (grades 6-12). The majority of the studies involved low-achieving, 
learning-disabled, or at-risk students. Subject matter in the majority of the studies was reading, either as 
part of English language arts or content-area instruction. Four studies focused on written or oral language; 
two studies focused on mathematics. The majority of studies were quasi-experimental with nonrandom 
assignment of students to treatment and control conditions. One study (Ketter & Pool, 2001) involved a 
qualitative research design. Finally, the majority of results were positive, that is, posttest scores were 
higher for students who received cognitively oriented instruction compared to those who did not. 

This review focuses first on the evidence of the effectiveness of cognitively oriented instruction 
on reading. Conclusions from a prior review of research are summarized, followed by a synthesis of 
findings from studies not included in the prior review. Implications for practice also are discussed. 
Findings and implications for practices are then discussed with regard to studies of the effectiveness of 
cognitively oriented instruction in writing and oral language and, finally, in mathematics. 

Cognitively Oriented Instruction in Reading 

In 1998, Dickson et al. reviewed 22 research articles on cognitively oriented instruction for 
improving the reading comprehension of diverse learners. Dickson et al. included eight theoretical and 
descriptive articles and 14 empirical studies. In five of the empirical studies, Dickson et al. found that 
low-achieving students, but not normally achieving students, benefited from instruction that taught 
students two or more metacognitive strategies, or a combination of metacognitive and cognitive (“how- 
to”) strategies. Instruction that taught both planning and self-regulatory strategies, or combined either 
planning or self-regulatory strategies with “how-to” strategies, had a positive influence on low-achieving 
students’ reading comprehension. The common “how-to” strategies found to be effective were 
summarizing, identifying main ideas, using visual imagery, and mapping elements and concepts of texts 
(Dickson et al.). 

Additionally, Dickson et al. (1998) found that for low-achieving students, teachers’ attention to 
two or more instructional components was more effective than attention to one component. For example, 
for remedial readers, modeling the target strategy plus providing feedback on the results of strategy use 
was more effective for enhancing reading comprehension than modeling alone (Schunk & Rice, 1992, as 
cited by Dickson et al.). Strategy-value feedback involves showing or telling students how performance 
on comprehension tasks is linked to productive or less productive strategy use. Consistent with Dickson’s 
et al. findings, Pressley and Harris (1990) claim that “students need to see evidence that the strategies 
they are learning really do lead to improved performance. Nothing motivates students to use a strategy 
like seeing that the strategy increases competent completion of an important task” (p. 33). Moreover, 
Dickson et al. concluded that to present combinations of strategies, implement multiple components of 
instruction, and for students to practice new strategies sufficiently, ample time must be allocated to 
cognitively oriented instruction in reading for low-achieving students. 
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For the present review, we found nine primary studies not included in or cited by Dickson et al. 
(1998). Analyses of findings from these studies confirmed Dickson et al.’s conclusions. 

Overview of Studies 

Table 3.2 summarizes the nature of the interventions and outcomes in the nine primary studies 
included. The studies in Table 3.2 are listed according to the direction of the results. Thus, from the top to 
the bottom of Table 3.2, studies with all positive results are listed first, followed by studies with mostly 
positive results, followed by studies with even results, followed by studies with mostly negative results. 

In each study, the direction of results refers to a comparison of the treatment group’s performance versus 
the comparison group’s performance on the posttest (where treatment involved participation in 
cognitively oriented instruction). 



Table 3.2. Content and Components of Cognitively Oriented Instruction in Reading for Low- 
Achieving or At-Risk Students 



Author(s) & Year 


Strategy Content 


Instructional Components 


Results" 


Dimino, Gersten, 
Camine, & Blake 
(1990) 


How to analyze story grammar and 
generate theme statements 


Purpose of strategy stated; strategy 
modeled; explicit explanations; peer- 
guided practice; sample theme 
statements given 


All positive 


Brown, Pressley, 
Van Meter, & 
Schuder (1995) 


Active reading strategies (identify 
genre; assess difficulty; predict/revise; 
ask & answer questions; visualize; 
summarize; problem-solve words) 


Instructional conversations about 
strategies; peer-guided and 
independent practice 


All positive 


Dermody (1988) 


Comprehension strategies 
(predict/evaluate; clarify; summarize) 
and analysis of task demands 


Modeling and explicit explanations; 
peer-guided and independent 
practice 


Mostly positive 


Brown (1995) 


Reading strategies (e.g., K-W-L; 
READ — read, encode, annotate, 
ponder) 


Peer mentors; new criterion 
identified and used to recognize high 
quality student work 


Mostly positive 


Jakupcak, Rushton, 
Jakupcak, & Lundt 
(1996) 


Use of mnemonics, concept mapping, 
RAP (Read, Ask questions; Put in own 
words), visualization, learning styles 
identified 


Could not be determined. 


Mostly positive 


Guastello, Beasley, 
& Sinatra (2000) 


Semantic mapping and use of map to 
study for test 


Modeling; explicit explanation; 
assisted practice; no independent 
practice 


Mostly positive 


McLain (1991) 


Predict/revise; self-monitoring 
understanding 


Checklist of steps and reminders 


Even 


Spiegel, Jackson, 
Graham, & Ware 
(1990) 


Mapping main idea & details 


No guided or independent practice; 
no evaluation of or feedback on 
strategy usage 


Mostly negative 


Waxman, deFelix, 
Martinez, Knight, 
& Padron ( 1 994) 


Verbalization, problem solving, 
imagery, graphic mapping 


Inquiry-based instructional 
approaches 


Mostly negative 



^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbmck (2000). 



Two of these studies met our criteria for high-quality studies (Dimino, Gersten, Gamine, & Blake, 
1990; Brown, Pressley, Van Meter, & Schuder, 1995). Four met our criteria for medium quality 
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(Dermody, 1988; Guastello, Beasley, & Sinatra, 2000; Spiegel, Jackson, Graham, & Ware, 1990; 
Waxman, deFelix, Martinez, Knight, & Padron, 1994). Three studies did not meet these criteria. In one of 
these studies (Brown, 1995), there was no comparison or control group, and thus, maturation could have 
explained the mostly positive gains from pre- to posttest. In the remaining two low-quality studies 
(Jakupcak, Rushton, Jakupcak, & Lundt, 1996; McLain, 1991), there was insufficient information 
reported to assess threats to statistical conclusions (e.g., no mean scores or standard deviations were 
reported, and the covariate was mentioned but not identified). The following analyses and review are 
based on the six studies meeting our criteria for medium or high quality. 

Findings 

A comparison of the top and bottom two studies presented in Table 3.2 suggests a distinguishing 
feature of effective cognitively oriented instruction in reading. In both Dimino et al. (1990) and Brown et 
al. (1995), with all positive results, but in neither Spiegel et al. (1990) or Waxman et al. (1994), with 
mostly negative results, the strategies taught included both task planning/preparation strategies and “how- 
to” strategies. In the Dimino et al. and Brown et al. studies, students learned to plan or prepare for an 
upcoming reading task by analyzing the story’s structure and/or identifying its genre and assessing task 
difficulty. Additionally, the students learned how to generate theme statements or ask and answer 
questions and summarize. In contrast, the strategies taught in the Spiegel et al. and Waxman et al. 
interventions included only “how-to” strategies (e.g., mapping and verbalization of ideas and details). 

This contrast in the content and results of the interventions match Dickson’s et al. (1998) conclusion that 
teaching students combinations of metacognitive and cognitive strategies brings about improved reading 
performance for low-achieving students. 

Moreover, the positive results of the Dimino et al. (1990) and Brown et al. (1995) studies were 
associated with interventions characterized by use of multiple instructional components. The instruction 
in the Dimino et al. study involved strategy modeling, explicit explanation of strategies, and student- 
guided and independent practice. The Brown et al. intervention, termed “transactional strategies 
instruction,” involved instructional conversations about strategy use and peer-guided and independent 
practice. Students reading below grade level participated in both of these studies. In both studies, the 
comparison groups received conventional reading instruction (i.e., an emphasis on phonics and skills for 
the second graders in the Brown et al. study and implementation of lessons and discussion questions as 
scripted in the teachers’ manual for the ninth graders in the Dimino et al. study). 

The effectiveness of the cognitively oriented instruction in the Brown et al. (1995) study may be 
attributed to two particular factors in addition to the nature of the strategy content and use of multiple 
instructional components. The duration of the intervention was long, lasting a full academic year; second, 
the teachers were highly experienced in using the approach, with at least three years prior experience 
using transactional strategies instruction. 

Importantly, in the Dimino et al. (1990) study, interaction effects were found. When posttest 
results were disaggregated by pretest achievement (measuring recognition of story grammar components), 
the positive results for the cognitively oriented instruction were found to be significant only for those 
students who performed low initially. Although all students were reading at least one year below grade. 
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only the students showing poor story grammar recognition, per se, benefited from this particular form of 
cognitively guided instruction. In practice then, these findings suggest that in order to accommodate the 
diverse learning needs of students, in addition to story grammar analysis, a variety of strategies should be 
taught, including, for example, theme generation, summarization, and asking and answering questions. 
Other ways to accommodate diverse learning needs include using a variety of text types, including 
narrative texts, informational texts, and other expository texts. 

Summary 

To summarize, in reading, research evidence indicates that effective cognitively oriented 
instruction has the following characteristics: 

1. Combinations of metacognitive and cognitive strategies are taught (e.g., planning 
ahead and summarizing). 

2. Multiple instructional practices are used (i.e., modeling and explicit explanation of 
strategies, instructional conversations about strategy use, and peer-guided and 
independent practice). 

The evidence also suggests that practitioners should take care when addressing the needs of low- 
achieving students not to ignore the needs of students who are proficient. To accommodate diverse needs, 
practitioners should teach multiple types of strategies and use multiple texts. Finally, one uncertainty 
remains. The effectiveness of complex cognitively oriented instruction in reading (such as Brown et al.’s 
[1995] transactional strategy instruction) when teachers try it out for the first or second time is unknown. 
More than a single year of practice and professional development may be needed before cognitively 
oriented instruction in reading positively impacts student achievement. This issue of sufficiency of 
teacher preparation and professional development needs to be addressed in future research. 

Cognitively Oriented Instruction in Writing and Oral Language 

Overview of Primary Studies 

With respect to writing and oral language, four studies addressed the effectiveness of cognitively 
oriented instruction for low-achieving students. These four studies are summarized in Table 3.3. Each of 
these four studies met our criteria for medium- or high-quality research. The O’Malley, Chamot, Stewner- 
Manzanares, Russo, and Kupper (1985) study focused on improving oral language, while the other three 
studies focused on improving writing. 

Findings 

O’Malley et al. (1985) conducted two experiments, one examining the effectiveness of 
cognitively oriented instruction for oral presentation skills and one examining the effectiveness of 
cognitively oriented instruction for listening comprehension. Participants in the O’Malley et al. study 
were English Language Learners (ELLs) performing at a beginning or intermediate level of English 
proficiency. In the oral presentation experiment, students were randomly assigned to one of three groups: 
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(1) metacognitive instruction and cooperative work groups, (2) cooperative work groups alone, or (3) a 
control group. Students in the metacognitive instruction and cooperative work group were taught 
functional planning, that is, they learned to identify the task demands of an oral presentation assignment, 
self-assess their knowledge of the language elements needed to meet those demands, and acquire the 
language elements they identified as missing. Additionally, in this group, students worked cooperatively 
rehearsing, providing corrective feedback, and polishing their oral presentations. Students in the second 
group worked cooperatively rehearsing, providing corrective feedback, and polishing their oral 
presentations, but were not given metacognitive instruction. Students in the control group were told to 
prepare oral presentations as they normally would. All students prepared multiple presentations about 
topics they selected from a list of six possible topics. 



Table 3.3. Content and Components of Cognitively Oriented instruction in Writing and Oral 
Language for Low-Achieving or At-Risk Students 



Author(s) & Year 


Strategy Content 


Instructional Components 


Direction of 
Results^ 


O’Malley, Chamot, 
Stewner-Manzanares, 
Russo, & Kupper 
(1985) 


Functional planning (identify task 
demands); self-assess; selective 
attention 


Peer review of draft presentations 
using specified criteria 


Mostly positive 


Englert, Raphael, 
Anderson, Anthony, & 
Stevens (1991) 


Analysis of text structure; self- 
regulatory questions (e.g., “What 
do I know?”); planning, drafting, 
and revising strategies with peer 
feedback 


Modeling, think aloud, think- 
sheets, instructional conversations 
about writing strategies; group, 
guided, and independent practice 


All positive 


Wong, Butler, Ficzere, 
& Kuperis (1996) 


How to plan, draft, and revise 
opinion essays 


Modeling, think aloud, plan-sheets, 
peer-guided practice 


Mostly positive 


Ketter & Pool (2001) 


Formulaic process (TAPF == Topic, 
Audience, Purpose, and Form); use 
of Think-abouts for idea generation 


Guided and independent practice, 
evaluation and feedback 


Mostly positive 



®The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



At posttest, the students who received the metacognitive instruction participated in 
cooperative work groups gave significantly higher quality oral presentations than the control group 
students (O’Malley et al., 1985). In particular, the former groups’ oral presentations were rated 
significantly higher on the dimension of organization as measured by more clear subordination and 
sequencing of ideas. Students in the cooperative work groups alone also produced higher quality oral 
presentations, but did not achieve higher ratings on organization. Thus, the combination of learning how 
to analyze and prepare to meet task demands plus opportunity to prepare and polish presentations with 
peer feedback resulted in significantly improved performance. 

In the O’Malley et al. (1985) listening experiment, students were randomly assigned to one of 
three groups: (1) metacognitive instruction in listening (i.e., use of selective attention) and cooperative 
work groups to verify what they heard, (2) cognitive strategy instruction in listening (i.e., notetaking) and 
cooperative work groups to verify their notes, and (3) a control group for which students were told to 
listen and prepare notes as they normally would for a lecture. On the listening comprehension posttest, 
neither the metacognitive nor the cognitive groups significantly outperformed the control group. 
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O’Malley et al. attributed the lack of positive impact to the difficulty of the listening tasks given the 
unfamiliarity of the content. Since the lecture was so unfamiliar to all of the student participants, the 
students derived little help from one another or from practicing selective attention and/or notetaking. 

The Englert, Raphael, Anderson, Anthony, and Stevens’ (1991) study examined the effects of a 
cognitively oriented strategy instruction approach that combined (1) planning/preparation strategies that 
involved analyzing and identifying text structures and task demands; (2) self-regulatory strategies with 
the aid of “think-sheets” (“Who am I writing for?” “What do I know?”); (3) group composition; (4) 
instructional conversations about writing problems and strategic solutions; and (5) peer sharing, editing, 
and feedback. This instruction was compared to district writing instruction emphasizing the recursive 
processes of brainstorming, planning, editing, and revising. Neither instructional conversations about 
problems and strategic solutions or peer sharing and feedback were included. 

Two different types of texts were studied and written: explanation texts and compare-and-contrast 
texts. On a posttest requiring students to compose an expert text, the cognitively oriented strategy group 
outperformed the control group even though neither group had been exposed to or taught how to write 
expert texts. The expert papers written by the strategy-trained students were particularly high on reader 
sensitivity scores. Englert et al. (1991) attributed this pattern to student participation in peer sharing, 
editing, and feedback. The public sharing of written compositions reinforced the value of and effort in 
writing interesting papers. 

The Wong, Butler, Ficzere, and Kuperis (1996) study also reported positive results associated 
with cognitively oriented strategy instruction in writing, but flaws in the designs threatened the validity of 
concluding that the cognitively oriented strategy instruction per se influenced the experimental students’ 
achievement. Without a novel task administered at posttest (i.e., a writing assignment that neither the 
strategy group nor the control group had practiced), Wong et al.’s study did not eliminate simple exposure 
to and familiarity with task demands rather than cognitively oriented instruction as a viable explanation 
for the results. An analysis of their findings, however, contributes information that helps develop our 
understanding of the effectiveness of cognitively oriented instruction. 

One of the five posttests used by Wong et al. (1996) is notable for its sole nonsignificant results. 
This particular posttest assessed students’ metacognitive awareness about writing, asking for example, 
“What things does a person have to learn to be a good writer?” and “What goes on in you head when you 
write?” (Wong et al., p. 201). At posttest, the students in this study who received the cognitively oriented 
instruction (instruction focused on “how to” plan, draft, and revise essays through modeling, plan-sheets, 
and peer-guided practice as listed in Table 3.3) nonetheless showed no more insight about the mental 
strategies involved in writing than students who did not receive cognitively oriented instruction. Notably 
lacking from Wong et al.’s instruction, but present in other effective instruction (i.e.. Brown et al., 1995; 
Englert et al., 1991), was opportunity to reflect on and engage in conversations about the strategies 
students were learning and applying. This contrast in content and results suggests that instructional 
conversations, in particular, may help low-achieving students develop self-awareness and insight about 
their roles and responsibilities as learners and academic performers. 
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The fourth and final study in this subcategory by Ketter and Pool (2001) is a qualitative study. 
They used in-depth interviews, observations, and writing sample analyses to examine the impact of 
cognitively oriented instruction on students’ interpretation of writing and developing proficiency. Ninth- 
grade students who previously failed the state writing exam participated in this study involving a 
semester-long course that taught students how to use a formulaic strategy (TAPF) and self-prompts 
(identify the “think-abouts”) to analyze and identify the task demands of a writing prompt and to plan and 
write a response that would meet criteria for passing. Ketter and Pool’s data illustrate how narrow the 
teachers’ and students’ interpretation of writing had become as they progressed through this test- 
preparation course. Although final student writing samples contained the requisite details and structure, 
they did not reveal the meaning of the experiences that were the topics of the essays, nor did the student 
writers establish relationships with their audience or explore relationships between ideas expressed in 
their essays (Ketter & Pool). The teachers were focused on “having students pass the test rather than on 
having students develop a wide range of writing skills” (Ketter & Pool, p. 384). Although these teachers 
were successful (98% of the 23 students passed the state writing exam), their instructional approach had 
its trade-offs. The unintended consequence appeared to be that students learned to write to pass a test, but 
failed to learn the value of any other kind of writing that required discovery of new meaning or 
understanding. Other observers have similarly identified trade-offs of standards-based reforms and 
accountability pressures as a vehicle for improving instruction provided to low-achieving or at-risk 
students, writing, “While high-stakes tests can motivate schools to try new strategies to raise student 
achievement, they may also provide a disincentive to adopting richer, more in-depth curricula” (AEL, 

2002, p. 3). 

Summary 

To summarize, evidence indicates that effective cognitively oriented instruction in writing and 
oral language for low-achieving students has the following characteristics: 

1. a combination of strategies for analyzing and preparing to meet task demands are 
taught; 

2. students are given opportunities to prepare, practice, and polish compositions and 
presentations with peer audiences and feedback; and 

3. instructional conversations are conducted allowing students to discuss task 
difficulties and possible strategic solutions. 

Furthermore, evidence indicates that there were trade-offs and/or unintended consequences of 
effective cognitively oriented instruction. Approaches that over-emphasized task analyses, use of 
cognitive strategies, and task success often did help students succeed, but appeared to do so at the expense 
of developing a deeper, more complex, multidimensional, and meaningful understanding of literate 
behavior. In the O’Malley et al. (1985) listening experiment, students were taught how to use selective 
attention and learned to take notes, but were not taught how to ask for clarification or what to do when the 
lecture content was unfamiliar. This approach, therefore, taught cognitive strategies in isolation, failing to 
acknowledge the academic content. Ketter and Pool (2001) reported a 98 percent success rate on the state 
writing exam, but student compositions lacked attention to meaningful relationships between ideas. In a 
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related reading study, the Dimino et al. (1990) study showed that although story grammar analysis 
instruction was successful for students who had specific difficulties with story grammar, the narrowly 
focused instruction and selection of reading materials failed to meet the needs of a number of other 
students in class. Taken together, these findings point to the need to check for and guard against 
decontextualizing strategy instruction and focusing too narrowly on remediating specific deficits. 

Cognitively Oriented Instruction in Mathematics 

In mathematics, cognitively guided approaches to instruction emphasize and support teachers 
taking an expansive view of mathematical knowledge and thinking by inquiring about and recognizing 
students’ diverse intellectual skills as they enter school or class. This emphasis derives from the 
conviction that “children from all kinds of backgrounds come to school with an impressive set of 
intellectual accomplishments” (Means & Knapp, 1991b, p. 286). What is needed is instruction that builds 
on these accomplishments and complements rather than contradicts what children already know and can 
do. To address this need, Peterson et al. (1991) developed an approach to mathematics instruction that 
involves researchers sharing with teachers research-based knowledge about children’s mathematical 
knowledge and thinking and letting teachers themselves interpret what this knowledge means for their 
instruction. This approach is called Cognitively Guided Instruction (CGI). Having teachers learn how 
students learn is a defining feature of CGI."^ 

More so than cognitively oriented instructional approaches to reading, the cognitively oriented 
approaches to mathematics utilize meaningful, whole, everyday tasks from outside the classroom. One 
way of making tasks meaningful for young children is to use their names in story problems. Another way 
is to bring everyday problems into the classroom and, from this context, pose questions such as, “Can you 
afford to build a kite frame given this kite plan, this amount of money, and this particular supply of 
wood?” (Bottge, 1999). Another distinguishing feature of CGI, in particular, and cognitively oriented 
instruction in mathematics in general, is recognition, encouragement, and exploration of multiple 
solutions through dialogue about solutions (Peterson et al., 1991). 

Peterson et al. (1991) summarize the findings from a dissertation study by Villansenor on the 
effectiveness of CGI in first-grade classrooms in inner-city schools. Villansenor compared first-grade 
students’ mathematics achievement over one year under two conditions. In one condition, children 
attended classes with teachers trained in CGI, and in, another condition, children attended classes with 
teachers trained in problem solving approaches to mathematics instruction. In both conditions, the schools 
served student populations with an average 79 percent eligible for free and reduced lunch. Children were 
administered pre- and posttests in problem solving and knowledge of number facts. Using the pre-test as a 
co-variate to adjust for initial differences in achievement, the students in the CGI condition significantly 
outperformed students in the comparison classrooms in both problem solving and recall of math facts. 



We are grateful to reviewer Robert Floden for making this point about CGI. 
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The CGI teachers in Villansenor’s (as cited in Peterson et al., 1991) study participated in a 
summer CGI workshop that involved deepening their understanding of different types of word problems, 
developing assessment practices, and planning how they would use their new knowledge in their 
classrooms (including listing problem types and developing a tentative unit plan). During the school year, 
the CGI teachers met once a month on Saturday mornings to share CGI ideas and to talk about 
implementing CGI in their classrooms. In contrast, control group teachers met twice during the same 
school year for a one-and-one-half-hour workshop on problem solving once in October and once in 
January. These contrasting teacher-learning opportunities in association with the positive direction of the 
CGI student results suggest a critical role for ongoing, discussion-oriented professional development with 
a focus on classroom implementation. Another mathematics intervention reviewed for the present chapter 
(Cardelle-Elawar, 1990) also utilized ongoing, discussion-oriented professional development that focused 
on classroom implementation. 

Overview of Studies 

Two studies examining cognitively oriented instruction in mathematics, but not CGI per se, were 
included in this review. The nature of the content and instructional components and the findings from 
these two studies are summarized in Table 3.4. In both studies, students were randomly assigned to either 
cognitively oriented instruction or control/comparison condition. Both studies met our criteria for 
medium- or high-quality research. 



Table 3.4. Content and Components of Cognitively Oriented Instruction in Mathematics for Low- 
Achieving or At-Risk Students 



Author(s) & Year 


Strategy Content 


Instructional Components 


Results“ 


Cardelle-Elawar 

(1990) 


Problem type identification; 
identification of key errors and error 
types (i.e., language error or 
computational error) 


Pre-test assessment and analysis of 
knowledge, errors, and strategies; finding 
exemplars of concepts in classroom 
environment; explaining pretest results to 
individual students; metacognitive coaching 


Mostly 

positive 


Bottge (1999) 


How to read, paraphrase, visualize, 
hypothesize, estimate, compute and 
check to solve word problems 


Use of contextualized, everyday problems; 
student work groups (dyads or triads); 
emphasis on multiple solutions 


Mostly 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Findings 

The Cardelle-Elawar (1990) study involved four teachers enrolled in a graduate course on 
mathematics for bilingual teachers. Each concurrently taught two sixth-grade bilingual mathematics 
classes. One of the two classes was randomly assigned to be the experimental, cognitively oriented 
instruction class; and one was assigned to be the control class. In the cognitively oriented class, teachers 
tried out instructional practices they were learning in the graduate course, and in their control class, they 
continued to teach in the manner in which they normally taught. The experiment lasted a total of six hours 
over a three-week period. Distinctions between the experimental and control classes are summarized in 
Table 3.5. 
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Table 3.5. Distinctions between Cardelle-Elawar’s (1990) Cognitively Oriented Instruction and 
Control Classes 



Feature 


Cardelle-Elawar (1990) Classes 


Cognitively Oriented Instruction 


Control 


Assessment 


Pre- and posttests administered and pretest 
results analyzed for key errors, which 
determined instruction 


Pre- and posttests administered, but 
pretest results not analyzed or used to 
determine instruction 


Strategy instruction 


Problem-embedded discussion and feedback 


Lecture and teacher-modeling at 
blackboard 


Corrective feedback 


Teacher inquired about and gave feedback 
on students’ strategies (e.g., “Which of 
these procedures you used is correct?”) 


Teacher said “wrong” or “right” 


Teacher role 


Guide 


Evaluator 



The defining features of Cardelle-Elawar’s (1990) cognitively oriented instruction were (1) use of 
pretest results to identify students’ key errors and lack of understanding problem types followed by (2) 
discussion and metacognitive coaching to guide students toward more sophisticated and self-aware 
selection and implementation of solution strategies. Teachers brought students’ pretest results to the 
graduate class and, with guidance from the instructor, analyzed the results and rehearsed new methods of 
providing feedback and instruction to students; subsequently, they implemented the new methods in their 
experimental classrooms. 

On Cardelle-Elawar’s (1990) posttest of problem solving, students in the cognitively guided 
instruction significantly outperformed the control students. These results are consistent with the findings 
of research on the effectiveness of CGI (Peterson et al., 1991) and show how ongoing professional 
development experiences for teachers that focus on discussing student work, mathematical thinking, and 
classroom implementation issues positively impact student achievement. 

The design of the Bottge (1999) study allows examination of the impact of embedding 
cognitively oriented instruction in meaningful problem contexts. Students were assigned to one of two 
treatments. One treatment was “anchored instruction,” which involved a semantically rich problem- 
solving environment presented on video {Bart's Pet Project) plus explicit instruction and guidance on 
how to use a five-step problem-solving strategy (paraphrase, hypothesize, estimate, compute, and check). 
The other treatment was “word problem instruction,” which involved explicit instruction and guidance on 
how to use the same five-step problem-solving strategy, but without the video context for the problems. In 
the “word problem instruction” condition, word problems were presented in typical textbook-like format. 
Bottge’s (1999) research also involved multiple methods of data collection. In addition to measuring 
achievement, students were observed during problem-solving posttests and interviewed about their 
learning experience and successes. 

On the transfer posttest (construction of wooden skateboard ramps), the “anchored instruction 
students” significantly outperformed the “word problem instruction” students (Bottge, 1999), supporting 
the conclusion that the combination of meaningful contexts for problem-solving and co^itively oriented 
instruction effectively helps students raise their achievement in mathematics. Computational performance, 
however, was not positively impacted, thus indicating that the benefits were specific to problem solving. 
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Student comments and observations confirm the importance of social context and interest to student 
success. Bottage highlights the experience of one student: 

[The student] explained that School math did not make sense to her and served no useful 

purpose. For her, the contextualized and applied problems did not look like math, which 

she regarded as a set of exercises unrelated to events in her life. (p. 91) 

Additionally, when students saw others in the class working on the problem, they became motivated, 
stopped arguing, and generated workable solutions. These observations reveal the potential of peer 
modeling for motivating effort and task initiation. When considering classroom applications of Bottge’s 
research, however, it is important to note that the “anchored instruction” and transfer posttest were made 
possible by the presence of certain resources at the research site, namely, the video-based vignette {Bart's 
Pet Project) and a technology education teacher with 20 years of experience who helped plan and 
supervise the posttest transfer problem involving construction of wooden skateboard ramps. 

Summary 

To summarize, three features of effective cognitively oriented instruction in mathematics were 
identified in the present review of research: 

1. Ongoing professional development for teachers involving discussions about student 
work and thinking in mathematics and a focus on classroom implementation issues. 

2. Combinations of cognitively oriented instruction with the use of contextualized 
learning activities that involve posing meaningful, relevant problems and arranging 
for peer groups rather than individuals to generate solutions. 

3. Incorporation of affective, motivational, metacognitive, developmental, and social 
features in the instruction. 

Summary of Findings across Studies of Cognitively Oriented 

Instruction 

Across the three areas or reading, writing and oral language, and mathematics, the majority of 
studies on cognitively oriented instruction reported positive results for low-achieving students. Several 
study designs allowed comparison of effects associated with different strategy content and instructional 
components. Findings suggest that combinations of metacognitive and cognitive strategies are more 
effective than either one type of strategy alone. Findings also suggest that teachers’ attention to several, 
rather than single, instructional components is more effective. 

In reading, student comprehension was aided with instruction that brought students’ attention to 
task demands and that taught, and had students practice, summarizing what they read. In writing and oral 
language, student performance was aided with instruction that combined instruction in strategies for task 
completion (including planning, drafting, and revising) with opportunities to share products and discuss 
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processes of writing with peers. In mathematics, student problem-solving achievement increased with 
instruction that utilized meaningful, contextualized problems, taught students how to prepare and solve 
problems systematically, and provided social contexts and peer modeling. 

Overall, the evidence reviewed in this chapter for the effectiveness of cognitively oriented 
instruction for low-achieving students is based on a small set of studies. Conclusions about the 
effectiveness of cognitively oriented instruction drawn from this review also are limited by time of 
posttest. None of the studies examined maintenance of teachers’ cognitively oriented instruction or 
student strategy use and achievement over longer periods of time than a couple of months or one 
academic year. More research is needed that examines the long-term benefits of cognitively oriented 
instruction. 



A Final Word to Practitioners 

Cognitively oriented instruction is designed to help students improve the quality of their thinking, 
become more independent learners, and become proficient in complex, higher order academic tasks. In 
this chapter, research on the effectiveness of cognitively oriented instruction for improving the 
achievement of low-achieving or at-risk students was reviewed. The results suggest that effective 
cognitively oriented instruction in classrooms is characterized as follows: 

• Teachers are modeling, explaining, prompting, and discussing combinations of 
metacognitive and cognitive strategies. 

• Students are modeling, explaining, prompting, practicing, and discussing 
combinations of metacognitive and cognitive strategies. 

• The content of instruction includes strategies for analyzing and preparing to meet 
task demands (e.g., problem or genre identification and identification of text structure 
elements). 

• The content of instruction includes “how-to” or formulaic, step-by-step solutions, 
procedures, and aids to comprehension. 

• Activity materials include a variety of text types and problem types. 

• Activity materials include texts and problems that are meaningful and/or relevant to 
students. 

• Teachers and students, and students and students, are engaged in two-way 
interactions in which preliminary products or solutions are discussed or shared and 
feedback on effectiveness is provided. 

• Students successfully perform complex, higher order academic tasks. 

• Students achieve at or above proficiency levels on tests involving complex, higher 
order academic tasks. 

Research suggests that when these indicators of effective cognitively oriented instruction are met, 
students benefit. Many students become interested in academic content and tasks, are engaged, and 
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successfully perform complex, higher order academic tasks. Readers are referred to Pressley et al. (1995) 
for practical guidance in how to implement cognitively oriented instruction. Effective implementation of 
cognitively oriented instruction, however, may require certain resources and organizational capacities, 
including (1) materials and technical know-how for creating authentic problem environments inside the 
classroom, and (2) ongoing professional development with external technical assistance providers or 
researchers that gives teachers opportunity to share and discuss student work and thinking and classroom 
implementation issues. Further research is needed that systematically examines the processes for securing 
such resources and their impact on classroom practices and student achievement. 

Finally, research also suggests that there are trade-offs or potential negative consequences to the 
implementation of cognitively oriented instruction for low-achieving or at-risk students. Teachers and 
education leaders need to be aware of and guard against focusing too narrowly on remediating strategic 
deficits to the exclusion of acknowledging, representing, and honoring the intellectual depth and breadth 
of content-area knowledge and skills (Ketter & Pool, 2001; O’Malley et al., 1985). 
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CHAPTER 4 

GROUPING STRUCTURES 



Kerry S. Englert 

Traditionally, schools have used tracking and other grouping strategies to help tailor instruction to 
the needs of diverse learners. Although tracking has allowed teachers to focus instruction on the needs of 
a homogenous classroom, it also has been problematic. The use of tracking has led to a lower standard for 
some students including English language learners, special education students, and a disproportionate 
number of minorities. Tracking also has resulted in a less rigorous and enriched curriculum, which has 
allowed many students to slip through the cracks academically (Oakes & Wells, 1998). 

Current thought relative to standards-based education is to increase learning for all students and 
to hold all students to the same high standards of learning and achievement . As a result, schools across the 
country are working to reduce the number of formal tracking programs. Students are in increasingly more 
diverse classrooms that include English language learners and special education students. This has created 
challenges for teachers who must find ways to tailor lessons to meet the needs of a diverse student body. 
The strategies presented below illustrate some methods teachers might use to improve student 
achievement for low-performing students in order to help them reach specified standards. 

Background 

For purposes of this research synthesis, the strategy of Grouping Structures refers to different 
ways of dividing classrooms into smaller groups of students for instructional purposes. Grouping 
strategies range from designing instruction for the specific needs of a small number of students of the 
same ability level to mixed-ability groups of students who work cooperatively. The research described in 
this chapter examines the potential of grouping structures for improving the achievement of low- 
performing students. 

There are many examples of grouping structures in education research. Slavin (1995) and Allen 
(1991) discuss methods such as comprehensive grouping, within-class grouping, and grouping for 
specific subject areas. Comprehensive grouping, which focuses on grouping students into ability-based 
classes, has similar goals to those of tracking. This type of grouping is typically implemented by schools, 
not by teachers, and is no longer a common practice in education. Therefore, this method is not discussed 
in the current synthesis. 

The two strategies of within-class grouping and grouping for specific subject-area instruction are 
strategies that can be implemented by teachers. Within-class grouping includes methods such as grouping 
students by ability within a class both homogenously and heterogeneously to differentiate instruction or to 
conduct cooperative learning groups. Grouping for subject-area instruction refers to grouping students for 
instruction in particular content areas. 
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Methodology 



Selection/Exclusion of Studies 

Although much research has been done to examine the impacts of various grouping structures on 
student achievement, this chapter examines only studies that provide explicit strategies to teachers on 
using grouping on regular classrooms, including mainstream classrooms. We excluded studies that 
addressed grouping for pull-out programs such as those used for Chapter 1 (currently Title 1) students and 
special education students because the regular teacher is not actively engaged in the instructional process 
after the students leave the classroom. We also excluded studies that incorporated multiple treatments 
where the effect of the grouping structure on student achievement could not be isolated. Based on these 
exclusions, we eliminated 43 of the 61 studies that the literature review produced. 

It is also important to note that seminal research studies on cooperative learning (e.g., Johnson, 
Johnson, Holubec, & Roy, 1984; Slavin, 1995) were not included because they fell outside the dates 
established for research synthesis (1985-current); they did not include achievement data or data on low- 
achieving or at-risk students; or they were commercial programs that were associated with a sufficient 
amount of research (e.g. Cooperative Integrated Reading [CERC]). For information on some of the 
research excluded from this synthesis, see Johnson et al. and Slavin. 

Subcategories within Grouping Structures 

The following sections describe research on the within-class and subject-area grouping strategies 
mentioned earlier. To increase clarity and to eliminate potential overlap, the two grouping structures have 
been reorganized into subcategories — grouping students by ability levels and grouping students 
heterogeneously or across ability levels. Examining the literature on grouping structures through these 
two lenses leads to insights into how these groups function differently in a classroom, what the issues are 
in implementing such groups, and how grouping impacts student achievement. Based on this information, 
educators can make more informed decisions about which grouping structures might work best in their 
classrooms. 



Grouping Students Across Ability Levels 

Ten studies addressed grouping students across ability levels. All the studies used quantitative 
measures. 

Overview of Studies 

A great deal of research has been conducted in the area of mixed-ability groups and specifically 
in the area of cooperative learning and its effectiveness in helping at-risk students (Waxman, Padron, & 
Arnold, 2001; Johnson et al., 1984; Slavin, 1996). Cohen (1994) defines cooperative learning in her 
review of small-group research. Cooperative learning, she maintains, consists of “students working 
together in a group small enough that everyone can participate on a collective task that has been clearly 
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assigned” (p. 3). It is important to note that Cohen’s definition stresses the need for participation of all 
students and clear assignment of the collective task. 

Researchers have identified other critical components of successful cooperative learning. These 
factors include the clarity of the directions that are given to students, the level of interaction between and 
among students, and the helping behaviors of the students (Webb & Farivar, 1994). The research studies 
presented in this section considered these and other factors in their research designs in order to examine 
the efficacy of cooperative learning. Other issues, such as teacher training in facilitating groups and the 
length of program implementation, also contribute to the effectiveness of the grouping structure. 

As Bossert (in Cohen, 1994) points out, many cooperative learning studies use a black-box 
approach and simply compare a cooperative learning approach to one that is non-cooperative. A research 
design such as this is deficient because it does not measure the impact of related factors that can influence 
the results. High-quality research studies of cooperative learning consider one or more of these factors. 
This characteristic, coupled with the quality criteria outlined in our coding protocol (see Appendix A), 
provided the rationale forjudging the quality of research studies on mixed-ability groups. Using these 
standards of quality, we coded four of the ten studies in this subcategory as high-quality studies. Although 
all 10 studies were considered as contributing to the evidence base supporting the overall impact of the 
intervention, more weight was given to the four high-quality studies by Henderson and Landesman 
(1992), Repman (1993), Webb and Farivar (1994), and, Yager, Johnson, and Johnson (1985). 

Findings 

The research reviewed indicates that mixed-ability grouping can be effective in increasing the 
achievement of low-achieving and at-risk students. Table 4.1 indicates the effectiveness of strategies 
based on the direction of findings from 10 studies. Results ranged from mostly positive to all positive, 
with only one study showing a negligible effect. This substantiates past reviews that consider the 
effectiveness of cooperative learning for all students (Webb, 1991; Slavin, 1996), which found that 
grouping students across ability levels produced generally favorable achievement results. 

A primary component to the success of mixed-ability grouping is the clarity of the directions that 
are provided to the students. Grouping works best when students are not given loosely defined tasks and 
roles (Repman, 1993): 

Structured activities specify roles and behaviors that students are to engage in during the 
collaborative learning session. Structure can be used to ensure that all group members 
participate in both giving and receiving help, as well as providing students with models 
of successful thinking and problem solving behaviors, (p. 151) 

By giving clear directions to students, teachers can ensure that students are more focused on the 
task and that they understand expectations for participation. Providing students with clear directions helps 
ensure that students of all ability levels have effective opportunities to learn the content and that they play 
active and important roles in the group. 
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Table 4.I. Characteristics of Studies Grouping Students Across Ability Level 



Author(s) 

&Year 


Treatment 

Size 


Grade 


Student 

Description 


Strategy 

Description 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Henderson & 

Landesman 

(1992) 


52 


7 


Low 

achieving 
and ESL 


Small collaborative 
learning groups and 
hands-on activities to 
help students make 
sense of math concepts 


Math 


2 grps pre- 
Post 

nonrandom 


Mostly 

positive 


Hooper & 

Hannafin 

(1988) 


40 


8 


Low 

achieving 


Computer-aided 
instruction with 
grouping and individual 
incentives 


Math 


2 grps post 
test random 


Mostly 

positive 


Klingner & 
Vaughn 
(2000) 


38 


5 


ESL 


Cooperative learning 
and helping behaviors 


Science 


1 grp pre- 
post test 


All 

positive 


Miller (1995) 


41 


8 


Low 

achieving 


Use of cooperative 
learning groups that 
were student led 


Social 

Studies 


2 grps pre- 
post 

nonrandom 


All 

positive 


Repman 

(1993) 


64 


7 


Minority 
and at risk 
of failure 


Collaborative 

computer-based 

learning 


Social 

Studies 


3 grps pre- 
post random 


Mostly 

positve 


Riley (2000) 


24/24 


6 


Low 

achieving 


Metacognition and 
strategic training in 
cooperative groups 
compared to direct 
instruction in 
cooperative groups 


Math 


3 grps pre- 
post 

nonrandom 


Mostly 

positive 


Singhanayok 
& Hooper 
(1998) 


46 


6 


Low 

achieving 


Cooperative learning 
using computers 


Science 


2 grps pre- 
post test 
nonrandom 


All 

positive 


Udupa(1993) 


10 


10 


Low 

achieving 


Concept mapping in a 
cooperative learning 
environment 


Science 


2 grps pre- 
post 

nonrandom 


Even 


Webb& 
Farivar (1994) 


91 


7 


Low- 
achieving 
and minority 


Cooperative learning 
with instruction and 
practice in basic 
communication and 
helping skills 


Math 


2 grps pre- 
post 

nonrandom 


Mostly 

positive 


Yager, 
Johnson, & 
Johnson 
(1985) 


25/ 25/25 


2 


Low 

achieving 


Three groups were 
studied: Cooperative 
learning that had 
structured oral 
discussion, cooperative 
learning that had 
unstructured oral 
discussion, and 
individual learning 


Lang. 

Arts 


2 grps pre- 
post random 


All 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Instructions to students also should consider students’ individual ability levels. Research on 
heterogeneous groups shows that students of different abilities assume different roles in a group. Higher 
achievers tend to provide more explanations; lower achieving students tend to ask more questions. Each 
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of these roles is critical to the learning process of the whole group. Low achievers are provided with 
information about the content in a timely manner, and higher achievers are able to solidify their own 
knowledge by responding to challenges and resolving disagreements (Henderson & Landesman, 1992; 
Webb & Farivar, 1994). Research indicates that cooperative learning can be an effective instructional 
strategy for both high and low achievers when adequately implemented (Singhanayok & Hooper, 1998; 
Yager etal., 1985). 

Research indicates that instruction may be especially important for low-achieving students 
because these students tend to be more passive during group learning situations (Repman, 1993). These 
students must be taught to ask clear and precise questions and to be persistent in getting needed 
information. With this type of instruction, students can more fiilly participate and obtain responses to their 
questions, which is important given the strong correlation between the degree of communication and 
student achievement in student groups (Webb & Farivar, 1994). Conversely, high-achieving students 
must be instructed that simply providing answers to a problem is not enough. They must elaborate on 
their explanations and make them “relevant, specific, timely, and clear” (Webb & Farivar, p. 371). Only 
then will the interaction benefit their peers. 

Yager et al.’s (1985) study also found a powerfiil group-to-individual transfer of knowledge in 
mixed-ability groups. When a question-and-answer dialogue was used as a cooperative leaning 
experience, all students (high, medium, and low ability) benefited. This research also examined the 
effectiveness of cooperative learning with and without training for students in structured oral participation 
and constructive listening. Students who performed the highest had been placed in cooperative learning 
groups and trained in structured participation. Yager et al.’s study reinforces the need for training to 
ensure that students have the skills they need to fiilly participate in mixed-ability groups. 

A final consideration when implementing mixed-ability grouping is the maturity level of the 
students. Eight out of ten of the studies examined implemented cooperative learning with students who 
were in middle school or higher. Teachers may find that using mixed-ability grouping with younger 
students is difficult because of the requirement placed on students to provide detailed explanations and to 
ask probing questions. Teachers might need to adjust the student training and grouping format to 
accommodate younger students. 

Grouping Students by Ability Levels 



Overview of Studies 

Eight research studies addressed grouping students by ability levels. Seven of the eight studies 
were quantitative; one study used mixed quantitative and qualitative research methods (Colby, Parker, & 
Wilson, 1995). It is worth noting that relative to the body of research on mixed-ability grouping, these 
studies were coded as being of lower research quality for a number of reasons. Four of the eight studies 
used small sample sizes, ranging from six to 12 students, for their treatment groups. Further, four out of 
the eight study designs did not use a control group, making it difficult to determine whether increased 
achievement was due to the student grouping structure or to student maturation. Based on these 
considerations, only one study (Bode, 1996) met the criteria for high quality. 
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In the studies in this subcategory, students were selected for a mixed-ability group based on 
teacher recommendations or their low test scores. Teachers then provided the groups of students with 
differentiated instruction in order to correct specific deficiencies. Instruction usually took the form of 
direct instruction or teacher-led group discussions. Across the eight studies, ability grouping was used 
most frequently in elementary schools. As indicated in Table 4.2, six out of the eight studies were 
conducted in lower grade levels. Only two were conducted at the middle school level, perhaps because 
elementary teachers are more apt to target students with limitations in specific content areas such as 
reading. 

Findings 

As previously mentioned, the group of studies in this section tended to be of lower quality than 
studies in the subcategory of grouping students across ability levels. However, there was one study coded 
as high quality that provided additional insight into the effects of ability grouping (Bode, 1996). This 
research was a secondary analysis of the Second International Mathematics Study (SIMS). The study 
connected background variables collected on the survey portion of SIMS to student assessment results. 
Using pretest and posttest scores, the study showed that ability grouping did not influence the 
achievement of low- or high-ability students. Although only one ability grouping study was coded as high 
quality, some of the other studies examined do provide insights into the effective use of ability groupings. 
Several of these studies reported that ability grouping increased achievement for low-performing students. 
The studies used a variety of instructional methods, but most targeted a specific skill. In six of the studies, 
teachers led the small groups and directed the instruction and student interaction patterns. For example, in 
a study by Paliscar, Brown, and Campione (1989), teachers read passages to groups of six students and 
then assessed listening comprehension by discussing the text with students. In studies by Hawley (2001) 
and Morris and Nelson (1992), supplemental reading instruction was provided and teachers used 
scaffolding to increase text difficulty throughout the year. Both studies reported positive effects when 
using ability grouping to improve the beginning reading and writing skills of low-achieving students. 

Several caveats should be considered when implementing ability grouping in the classroom. In 
her research synthesis on ability grouping and tracking, for example, Lindle (1994) points out that the 
practice of ability grouping is not a panacea. She suggests that segregating students by ability might result 
in lower student self-esteem and might impact achievement by resulting in lower expectations rather than 
“enriched, differentiated instruction” (p. 8). In addition, trends in cognitive psychology refute the tenets 
on which the theories for ability grouping are based. In particular, some uses of ability grouping reflect 
the notion that students must master basic skills before moving on to problems involving higher order 
thinking. However, as Lindle states, “skills have no prerequisites to content” and that “skills have no 
meaning unless they are tied to content” (p. 5). 

In the studies involving students in higher grade levels, student direction of the groups became 
more prevalent than in lower grades. Harris, Dyer, and Tracz (1988) reported some positive effects for 
low-achieving students by training them in study skills such as vocabulary, report writing, and grammar 
using cooperative learning methods. Students were told to apply these methods in their academic subjects. 
However, some complications can occur when applying these methods. As already discussed, cooperative 
learning tends to employ heterogeneous groupings of students, and in order for cooperative learning to be 
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effective, teachers must structure the cooperative learning sessions rather than expecting students to apply 
it themselves (Webb & Farivar, 1994). 



Table 4.2. Characteristics of Studies Grouping Students by Ability Level 



Author(s) 
& Year 


Treatment 

Size 


Grade 


Student 

Description 


Strategy 

Description 


Subject 

Matter 


Type of 
Comparison 


Results" 


Bode (1996) 


3991 


8 


Low 

achieving 


Large-scale 
study looking at 
effects of 
general 
grouping 


Math/ 

Science 


2gps pre-post 
nonrandom 


Even 


Bruce, 
Snodgrass, 
& Salzman 
(1999) 


11 


1 


Low 

achieving 


Guided reading 
in small groups 


Reading 


Igrp Pre-post 


All 

positive 


Colby, 
Parker, & 
Wilson 
(1995) 


6 


3 


Below grade 
level in 
reading and 
writing 


Mini-writing 
lesson, free 
writing, and 
group sharing 


Writing 


1 grp pre-post 


All 

positive 


Harris, 
Dyer, & 
Tracz 
(1988) 


79 


8 


Low 

achieving 


Taught study 
skills using a 
cooperative 
learning 
approach 


Study skills 


2 grps post 
random 


Mostly 

positive 


Hawley 

(2001) 


12 


K 


Low SES 
and low 
achieving 


Small-group 
instruction in 
phonological 
and phonemic 
awareness and 
literacy 
acquisition 


Reading 


1 grp pre-post 


All 

positive 


Morris & 

Nelson 

(1992) 


10 


2 


Low 

achieving 


Supplemental 
instruction 
provided by 
teacher 


Reading 


2 grps pre- 
post 

nonrandom 


Mostly 

positive 


Palinscar, 
Brown, & 
Campione 
(1989) 


36 


1 


Low 

achieving 


Teacher- led 
reading and 
student 
discussion of 
passages 


Reading 


2 grps pre- 
post 

nonrandom 


All 

positive 


Welch, 

Richards, 

Okada, 

Richards, & 

Prescott 

(1995) 


600 


4-6 


Low 

achieving 


Paraprofessional 
under teacher 
supervision 
provided direct 
instruction in 
cross-grade 
groupings 


Reading and 
Math 


2 grps pre- 
post 

nonrandom 


Mostly 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 
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Summary of Findings Across Grouping Structures 



This research synthesis on Grouping Structures examines two very different approaches to 
grouping students — heterogeneous, or mixed-ability grouping, and homogenous, or ability-level 
grouping. Studies of both of these grouping interventions report positive impacts on low-achieving 
students or at-risk students. All but two studies included in this chapter produced mostly positive to 
positive results. The two remaining studies found no influences of groups on student achievement. This 
indicates that as teachers work in increasingly diverse classrooms, the application of small groups might 
be one way to modify instruction to increase student achievement. 

With regard to mixed-ability groupings, results of the high-quality studies were consistent. All of 
these studies reported mostly positive to positive results on the effects of grouping on student 
performance. In particular, cooperative learning, when rigorously implemented, can provide students with 
enriched instruction through peer interaction resulting in improved student achievement. It is important to 
bear in mind that the research also indicates the importance of training and professional development on 
the successful implementation of this classroom practice (Webb, 1991; Repman, 1993). Research studies 
stress the importance of the processes that occur in groups and of the teachers’ role as critical to student 
learning in groups. 

The lack of high-quality research limits our ability to make confident statements about the 
efficacy of ability grouping. As a whole, the research on ability grouping shows that there may be positive 
outcomes associated with this intervention, although the one high-quality study in this subcategory found 
neither positive nor negative effects on student achievement and thus does not support the favorable 
findings reported by the lower quality research studies reviewed. 

A Final Word to Practitioners 

The studies reviewed in this chapter demonstrate the effectiveness of applying different grouping 
structures in classrooms of low-achieving students. These studies, and in particular the high-quality 
studies, showed that there are several critical components of successful group learning. When forming 
groups across ability levels, such as cooperative learning groups, important factors include the clarity of 
the directions given to students, the level of interaction that exists between and among the students, and 
the helping behaviors of the students (Webb & Farivar, 1994). Other mediating factors, such as teacher 
training in facilitating groups and the length of program implementation, also contribute to the 
effectiveness of the intervention. 

Grouping by ability level was the second type of grouping structure discussed in this chapter. The 
one high-quality study of ability grouping was a secondary analysis of a large-scale dataset, but it did not 
include program descriptions that could be of use to practitioners in implementation. The other studies of 
ability grouping are not of sufficient rigor to make conclusive recommendations about effectiveness. 

Teachers who are considering implementing a particular grouping strategy should consult 
additional resources. Cooperative learning is an instructional approach supported by a large amount of 
general research and with specific programs that support its use. One comprehensive source of 
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information is Cooperative Learning: Theory, Research, and Practice (1995), by Slavin, a seminal 
researcher in this area. The author shares theoretical information that helps explain why cooperative 
learning works. In addition, he translates the research into practical guidelines for the most widely used 
cooperative learning strategies with step-by-step instruction for implementation. 
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structured collaboration, students were given a protocol sheet to promote and direct elaboration of 
responses and questioning. The students in treatment group 3, structured collaboration with training, were 
given explicit instructions in providing elaborative explanations to other students’ questions and 
providing explanations when errors occurred. Results showed that differences in achievement occurred 
between treatment group 1 and the combined results of treatment groups 2 and 3. Thus, students who 
were given instruction or information on elaborated responses scored higher on the posttest. 
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The goal of this study was to examine the impact on achievement of providing instruction to 
students in helping skills. Classes were randomly assigned to the treatment or comparison group. The 
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comparison group was given only instruction in basic communication. This study found that minority 
students were more likely to received and give help in the treatment group. These students also performed 
better on the mathematics posttest than did their peers in the comparison group. However, their gains 
were not enough to close the achievement gap with the White students. 
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Researchers examined the differential effects of two types of cooperative learning: structured and 
unstructured oral discussion. The researchers were attempting to address if there was a group-to- 
individual transfer of knowledge and, if so, under what conditions. The members of the structured 
cooperative learning group were given role assignments to help direct their participation in the group 
assignment. The unstructured oral discussion group was told only to collaborate and the students were not 
given specific roles to play. The students in the control group were placed in an individualistic learning 
situation where the students’ achievement goals were independent of the other students. Researchers 
found that students in the both cooperative learning groups did better on achievement tests immediately 
after the treatment and 18 days later. Their analyses also showed that the structured cooperative learning 
group performed higher than the unstructured group. The achievement gains in both treatment groups 
were consistent across ability levels. 
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CHAPTER. 5 



TUTORING 

David Snow 

As schools address the problem of bringing low-achieving students up to proposed standards, 
they are forced to decide how best to use limited resources. Among the more aggressive approaches to the 
problem are interventions that involve tutoring. The money, time, space, and effort that schools dedicate 
to tutoring tax limited school resources, but these efforts are the most attentive means to address the needs 
of at-risk students. The good news is that research brings to light a number of program variations that ease 
the need for significant resources. This chapter reviews tutoring research that focuses on improved 
achievement for low-achieving students, describing program characteristics as well as evidence of 
success. 



Background 

For the purpose of selecting the studies for this chapter, tutoring is defined as a one-on-one 
interaction between tutor and student. The tutors in the programs studied varied widely, from children to 
retirees, but in most cases the tutees were young readers. Studies of cross-age student tutoring 
interventions were included if the tutor was not expected to gain academically from the interaction. 
Tomlin (1995), for example, describes a tutoring program that paired academically strong high school 
students with at-risk middle schoolers. Since improved academic achievement was not expected from the 
high school tutors, this study was included in the chapter. Note that studies examining cross-aged tutoring 
for mutual curricular gain are not included here. These cross-aged tutoring studies are synthesized along 
with other peer tutoring research in Chapter 6. 

Methodology 

Two recent syntheses of tutoring studies preceded this synthesis. In the first, Fager (1996) lists a 
number of positive outcomes that she attributes to tutoring programs: encouraging community 
relationships, improving self-esteem and confidence, and increasing achievement. These outcomes were 
drawn from a review of prior research, which included studies from 1988 to 1995, and were reinforced by 
the author’s own study of seven tutoring programs in the American Northwest. Fager does not report 
quantitative evidence of program success such that comparisons could be made or the level of success 
predicted, but the work includes citations and is a good resource for practitioners. 

The second synthesis is more relevant to this paper. In a meta-analysis of tutoring program 
research, Elbaum, Vaughn, Hughes, and Moody (2000) quantitatively combined results of reading 
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program studies published between 1976 and 1998 that utilized adult tutoring interventions to improve 
the achievement of at-risk students. As a result, the authors report a combined effect size^ of more than 
0.5 standard deviations for the tutored students. They suggest that a well-designed, carefully implemented 
program need not be staffed by trained professionals in order to be effective. Because the work of Elbaum 
et al. is current and thorough, we see no need to reconstruct their meta-analysis. Instead, we separated our 
pool of tutoring studies into three small, homogenous subcategories: professional tutoring, volunteer 
tutoring, and student tutoring. This grouping of studies allows us to best identify common issues most 
important to tutoring program implementation. 

Selection/Exclusion of Studies 

Of the 3 1 studies identified as tutoring research after the initial screening described in Chapter 1, 
eight additional studies were excluded here. Three were not research studies, and five of the studies 
simply did not report enough methodological information to support their inclusion. Thus, 23 studies have 
been included in this chapter’s synthesis. These studies are a collection of reported research and program 
evaluations, each revealing unique perspectives on issues related to tutoring and the effect of tutoring 
programs on at-risk students. 

It is important to note that five of the studies reviewed in this chapter are works that were coded 
as high in quality. These studies had random assignment of students in thoughtful experimental designs, 
careful descriptions of implementations, and thorough quantitative and qualitative analyses. These 
attributes increase confidence in reporting the effects of tutoring programs. Thus, we rely heavily upon 
the work of McCarthy, Newby, and Recht (1995), Cobb (1998, 2001), Baker, Gersten, and Keating 
(2000), and Meier and Invemizzi (1999) in drawing conclusions. 

Subcategories within Tutoring 

The sections of this chapter synthesize the 23 tutoring studies in three groups. First, studies of 
professionals as tutors are presented (n = 5). This group of studies examines the most qualified and least 
affordable pool of tutors: certified teachers, reading specialists, and paraprofessionals. The McCarthy et 
al. (1995) study that was coded as high quality is included in this section. The second group of studies is a 
set analyzing adult volunteer tutoring programs (n = 13). This set includes programs that recruit tutors 
from the ranks of retirees, college students, and other civic-minded adults in school communities. The 
remaining four studies that were coded high quality (Cobb, 1998; Cobb, 2001; Baker et al., 2000; Meier 
& Invemizzi, 1 999) are included in this section. The last group is a small group of studies that examine 
cross-age tutoring programs (n = 5). Several important issues result from these studies of the most readily 
available tutors. None of the studies in this last group was coded as high in quality. 

In each section of this chapter, the issues surrounding the tutoring programs will be reported, 
along with a review of program effects. 



^The effect size is the difference between the treatment and comparison groups expressed in standardized 
units. An effect size of .3 or greater is considered large enough to be practically meaningful. 
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Professional Tutoring 



Tutoring in schools is a variation of the long-standing apprenticeship and mentoring practices 
viewed as successful in a variety of professions. The use of experienced professionals in the tutoring roles 
— master plumbers and veteran doctors being good examples — gives strength to the notion that 
professional teachers are best suited for the role of tutoring. The tutors in this subgroup are such a set of 
professionals: licensed teachers, trained specialists, and the like, A review of these studies reinforces the 
synthesis reports of the effectiveness of tutoring programs (Elbaum et al., 2000; Wasik, 1998; Wasik & 
Slavin, 1993) and emphasizes the effective characteristics of tutoring. 

Overview of Studies 

The five studies presented in Table 5. 1 represent recent, more rigorous research on programs that 
provide professional tutoring for students at risk for academic failure. The table displays the varying 
professional status of the tutors, from aides to certified teachers, while also emphasizing the fact that the 
tutees were a far more homogeneous group: all very young, at-risk readers. The ages of the students 
probably reflects recent program and research emphasis on early literacy, and the small number of studies 
is likely a reflection of the limited use due to the expense of professional tutoring efforts. 



Table 5.1. Characteristics of Studies involving Professional Tutors 



Author(s) &Year 


Treat 

ment 

Size 


Tutors 


Grades 

of 

Tutees 


Student 

Description 


Subject 

Matter 


Type of 
Comparison 


Results" 


Johnson (1987) 


1,417 


trained 

adult 

aides 


K, 1 


low test scores 


reading 


pre-post 
treatment only 


Mostly 

positive 


Mantzico-poulos, 
Morrison, Stone, & 
Setrakian (1992) 


111 


trained 

teachers 


K 


low test scores 


reading 


3 grps pre-post 
random 


Even 


Matz 1989 


1 


researcher 


young 

child 


teacher-identified 
low performer 


reading 


case study 


All 

positive 


McCarthy, Newby, & 
Recht(1995) 


19 


trained 

specialists 


1 


teacher-identified 
low performer 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Morris, Tyner, & 
Pemey (2000) 


43 


trained 

teachers 


1 


low test scores 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Findings 

Despite the small number of studies in this subcategory, there are several emerging conclusions 
from this group that are worth mentioning. McCarthy et al. (1995), for example, reported lasting effects of 
a first-grade reading program. These researchers found significant differences between a tutored group of 
students and a control group two years after the completion of the program. These results are consistent 
with those from a large-scale program synthesis conducted by Wasik and Slavin (1993). However, 
McCarthy et al. note that sustained gains were accomplished without the added expense commonly 
associated with large-scale efforts. These two observations of lasting effects provide evidence that early 
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tutoring intervention, whether in the form of large- or small-scale programs, can be effective with at-risk 
readers. 



As noted earlier, strong theoretical backing was used in the design of the tutoring programs 
analyzed by Mantzicopoulos, Morrison, Stone, and Setrakian (1992). Each treatment was a series of 
methods driven by differing reading theories, and the authors reported that the purposive aim of the 
tutoring program gave the effort needed support. Similarly, Matz (1989) describes stages of intervention 
influenced by a variety of common methodologies, and Johnson (1987) describes tutoring sessions 
influenced by Direct Instruction. In each of these studies, a strong purposive implementation in the 
program being studied led to increased achievement. 

One study also tied program success to thorough, specific tutor training and continued program 
monitoring. Morris, Tyner, and Pemey (2000) describe Early Steps, an early intervention reading program 
similar to Reading Recovery (a large-scale methodology) in terms of its careful tutor training and the 
formative evaluation of the tutoring sessions throughout program implementation. 

Most prevalent in this group of studies is the conclusion that tutoring is characterized by thorough 
and frequent diagnostic and prescriptive interchanges between tutor and tutee. McCarthy et al. (1995) 
conducted a thorough study of a program adapted from an approach used in Reading Recovery. Four 
credentialed tutors were trained by the lead researcher to promote reading fluency and phonological 
awareness prior to program implementation. At the heart of the tutoring program, however, were the 
weekly meetings throughout the intervention between the researchers and the tutors in which the 
pedagogical training continued. In these sessions, the students’ performances were used to guide the 
tutors in their approaches to instruction. Matz (1989) described smaller and more frequent diagnoses in 
his successful experience in tutoring a young reader by means of a series of informal evaluations that 
prompted individualized approaches in his teaching. Whether formal or informal, it is apparent in these 
studies, as well as others (in fact, each of the studies listed describes active diagnostic/prescriptive 
interaction), that the diagnostic-prescriptive function of the tutor is essential to taking full advantage of 
one-on-one instruction. The existence of this complex interaction in tutoring programs encouraged 
Johnson (1987) to suggest more careful consideration of the diagnostic-prescriptive process in program 
design. 



The importance of this interaction, therefore, suggests that trained tutors are best suited for the 
role. As McCarthy et al. (1995) note, “instruction was individually shaped by the teacher in the role of the 
expert evaluator” (p. 288). This expertise, the authors suggest, was instrumental in producing the strong 
effects they reported on a long list of reading and writing measures (effect sizes ranging from 0.50 to 0.79 
for experimental variables). Within this group of studies, the tutors’ expertise was called upon repeatedly 
to evaluate skills and to adapt instruction. The Early Steps program studied by Morris et al. (2000) 
expected professional teachers to rely on their understanding of instruction in word families, vowel 
patterns, and complex contrasts while tutoring first-grade readers. Mantzicopoulos et al. (1992) described 
a more complex program in which the tutors delivered instruction through visual, motor, auditory, body 
image, and intermodal means. Specifically, “the visual cluster consists of visual discrimination and recall, 
visual sequencing, and visual figure-ground tasks, all considered essential to reading” (p. 578). These 
authors imply that tutor expertise is nearly essential to the task of tutoring. 
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It is clear that the implementers rely upon tutor expertise, but is such expertise necessary? In his 
study of professional tutors, Mantzicopoulos et al. (1992) reported significant gains in only one of the five 
instructional modes listed above. Although the other studies in this subcategory reveal more positive 
results, we cannot conclude that the level of training exhibited by the tutors in the other studies is 
necessary for the achievement of the tutees. This issue is explored further in the sections to follow as the 
adult volunteer and cross-age student studies are analyzed and results are compared. 

Volunteer Tutoring 

Any societal notion that a school is an institution working in isolation within the larger 
community is a notion challenged by programs that gather community volunteers and enlist them as tutors 
for school children. This subgroup of studies examines tutoring programs that do not carry the expense 
associated with the professional tutoring programs described above. These works provide a rich sense of 
the volunteer programs, their relevant issues, and the program effects. Note that most of these studies 
describe their volunteers in the strictest sense, giving their time for merely intrinsic rewards, but this 
collection also includes studies of programs that offer tutor stipends and those that provide college credit 
for tutoring work. 

Overview of Studies 

The studies in this group are again characterized by small samples of students exposed to early 
literacy interventions. Four of these studies are coded as being high in quality: the recent quantitative 
studies by Baker et al. (2000), Cobb (2001), and Meier and Invemizzi (1999), and the qualitative work of 
Cobb (1998). We are able to draw preliminary conclusions based on these studies with supporting 
evidence provided by the other volunteer tutoring studies in this group (see Table 5.2). 

Findings 

The Baker et al. (2000) study of SMART, a volunteer tutoring program implemented in Oregon, 
was one with a larger number of students studied. Among the results of this work is a set of reported 
effect sizes that are comparable to those reported in Elbaum et al. (2000) and Wasik and Slavin (1993). 
The SMART volunteer tutors found effect sizes ranging from .32 to .53 on a series of five different 
reading measures. Can volunteers be as effective as professional teachers when serving as tutors? The 
results of the SMART study, supported by other studies in this set, indicate that positive achievement 
results from tutoring programs do not rely on professional tutors. 

The group of studies in this section reinforces some programmatic characteristics that were noted 
in the previous section. The importance of a purposive treatment is one such example. We consider a 
strong “purposive treatment” to be any instructional framework that is used by tutors to guide instruction. 
We found no studies that compared purposive treatments to a more casual approach, but there are several 
examples of studies in this subcategory in which a well-defined purpose seemed to be central to the 
tutoring effort. 
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Table 5.2. Characteristics of Studies involving Adult Volunteer Tutors 



Author(s) &Year 


Treat 

ment 

Size 


Tutors 


Grades 

of 

Tutees 


Student Description 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Baker, Gersten, & 
Keating (2000) 


43 


adults 


1.2 


teacher-identified 
low performer 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Cobb (1998) 


4 


College 

students 


4 


low test scores 


reading 


case study 


All 

positive 


Cobb (2001) 


30 


College 

students 


2,3 


teacher-identified 
low performer 


reading 


2 grps pre-post 
random 


Even 


Juel(1991) 


1 


College 

students 


1 


teacher-identified 
low performer 


reading 


case study 


All 

positive 


Knapp & Winsor 
(1998) 


8 


adults 


2,3 


teacher-identified 
low performer 


reading 


2 grps pre- 
postnon random 


Mostly 

positive 


McCarthy, Newby, 
& Recht (1995) 


4 


Senior 

citizens 


1,2,3 


teacher-identified 
at-risk. Learning 
Disabled, or Dyslexic 


reading 


case study 


All 

positive 


Meier & Invemizzi 
(1999). 


28 


Senior 

citizens 


1 


low test scores 


reading 


2 grps pre- 
postrandom 


Mostly 

positive 


O'Sullivan, Puryear, 
& Oliver (1994) 


16 


College 

students 


9 


teacher-identified 
low performers 


all 


2 grps pre- 
postnonrandom 


Even 


Ramey (1990) 


8 


adults 


2,3,4, 

5 


identified to be in need 
of services 


reading 


4 grps pre-post 
nonrandom 


Even 


Ramey (1991) 


77 


adults 


2, 3, 4, 
5 


identified to be in need 
of services 


reading 


4 grps pre- 
postnonrandom 


Mostly 

positive 


Richardson, 
Abrams, Byer, & 
DeVaney (2000) 


24 


College 

students 


7, 8, 9, 
10,11, 
12 


teacher/district 
identified as “being in 
need of reading 
assistance” 


reading 


1 grp pre- 
postnonrandom 


Even 


Rimm-Kaufman, 
Kagan, & Byers 
(1999) 


21 


senior 

citizens 


1 


teacher nomination 


reading 


2 grps pre- 
postrandom 


Even 


Vadasy, Jenkins, 
Antil, Wayne, & 
O’Connor (1997) 


20 


adults 


1 


low test scores 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Cobb (2001) describes a simple yet clear programmatic aim in experiences that strove to make 
curricular connections using puppets and games. The absence of results from this study was attributed to 
the small number of subjects, a problem seen throughout the research on tutoring. Richardson, Abrams, 
Byer, and DeVaney (2000) report the use of an interest inventory taken by the tutees therefore 
customizing the upcoming tutoring sessions as well as providing a guiding purpose for the interactions. 
The challenge of providing a clear aim was eased in the successful SMART Program mentioned above 
(Baker et al., 2000). The authors describe a successful tutoring program in which the tutoring curriculum 
was intentionally independent of the students’ classroom work. By not struggling to directly support the 
classroom teachers’ lessons, the tutors were apparently freed to concentrate on a simpler set of guidelines. 

As noted under Professional Tutoring, constant supervision and regulation of the tutoring sessions 
was again a reoccurring issue in this literature. Knapp and Winsor (1998) report that program supervisors 
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worked with tutors to adjust the reading level of books on a regular basis. Vadasay, Jenkins, Antil, 

Wayne, and O’Connor (1997) and Rimm-Kaufman, Kagan, and Byers (1999) also describe programs with 
thorough formative assessment of their tutoring sessions, while Meier and Invemizzi (1999) conclude 
their work with a call for “intensive, ongoing training and supervision of tutoring under the guidance of a 
knowledgeable reading specialist” (p. 332). 

Additional issues come to light as a result of comparing these studies. The logistical quality of 
volunteer tutoring programs is one strong concern. The studies in this section describe logistics of 
impressive quality, like a program that provided its tutors with tote bags filled with classroom supplies: 
dry erase boards, tablets, pencils, markers, and manipulatives (Cobb, 2001). But poor logistical quality 
also was reported. For example, O’Sullivan, Puryear, and Oliver (1994), in their study of a program 
utilizing college-aged tutors, explain that the tutors were not made to feel welcome in the host high 
schools. 



The strongest issues, however, seem to center around the volunteer tutors themselves. One 
important question is where these tutors are to be found. Ranks of retirees and college students seem to be 
the most frequent sources. One of the described programs managed to recruit a number of retirees who 
were at one time professional teachers (Rimm-Kaufinan et al., 1999), while another used college-age 
student athletes who had experienced trouble in school themselves (Juel, 1991). Retirees and college 
students are not the only tutors described in these studies however. The programs described several 
creative approaches to finding willing and patient volunteers: parents and grandparents (Vadasay et al., 
1997), preservice teachers (Cobb, 1998), and working adults (Baker et al., 2000; Meier & Invemizzi, 
1999). 



In programs where competent volunteers could be found, trained, and retained, the reported 
effects were in several cases as large as those programs employing professional teachers as tutors. 
Returning again to the report of the SMART Program (Baker et al., 2000), the authors report a 50 percent 
annual retention rate in the program tutors. This retention rate, the authors claim, is one good reason for 
the success of the program and the reason that the program is cost-effective. Accepting this claim 
suggests that finding and retaining good tutors may be the most important issue faced by volunteer 
tutoring programs. In any case, it is clear that providing training that is appropriate for the wide variety of 
tutors in these studies is a creative challenge. 

Compared to the cost of implementing a tutoring program using professional teachers, the cost of 
using volunteer tutors is quite low. This is not to say that the effort is cost free. In an investigation of 
volunteer tutoring programs, Wasik (1998) reviews program characteristics, including information about 
specific program implementation costs. 



Student Tutoring 

Five studies fall into this grouping of the use of student tutors. These studies describe programs in 
which the students’ tutors were significantly older or otherwise advanced beyond the skill levels of the 
tutees. Unlike the peer tutors described in Chapter 6, these tutors are not expected to gain academically 
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from the tutoring interactions. Although small in number, this group of studies does help to inform the 
issues already presented. 

Overview of Studies 

As reported in the previous sections, the five student tutor studies in this section illustrate the 
research focus on early literacy (see Table 5.3). The programs studied employed tutors of varying ages; 
most of the programs resulted in positive effects. We were again limited in our ability to make 
generalizations from these works due to a small number of tutored treatment samples, although these 
studies do introduce new issues and support conclusions draw from the previous sections. 



Table 5.3. Characteristics of Studies involving Cross-Age Student Tutors 



Author(s) &Year 


Treat 

ment 

Size 


Grades 

of 

Tutors 


Grades 

of 

T utees 


Student 

Description 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Harris, Marchand- 
Martella, & Martella 
(2000) 


88 


11, 12 


9 


low test scores 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Jenkins, Jewell, 
Leicester, Jenkins, & 
Troutner (1991) 


15 


inteime 

diate 


1,2,3 


low test scores 


reading 


2 grps pre-post 
nonrandom 


Even 


Taylor, Hanson, 
Justice-Swanson, & 
Watts (1997) 


19 


4 


2 


teacher-identified 
low performer 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Tomlin (1995) 


36 


high 

school 


6,7,8 


documented at-risk 
(school records) 


all 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Zukowski (1997) 


1 


5 


3 


teacher-identi fied 
low performer 


reading/ 

writing 


case study 


All 

positive 



“The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Findings 

Student tutors are certainly a readily available group. Taylor, Hanson, Justice-Swanson, and 
Watts (1997) studied a program that drew fourth-grade tutors from within the school to work with second 
graders. Their availability was somewhat tempered, however, by the work needed to coordinate schedules 
between the two classes. It is not clear what the tutors in this program were expected to gain from the 
experience, and it is important to consider the instructional time these older students were missing during 
their school day. For this reason, it is likely that programs of this type tend to be scheduled outside the 
school day, and this observation may well explain the lack of available school-day research on at-risk 
tutoring interventions. 

Regardless of the logistics, there does seem to be potential in this type of program. Zukowski 
(1997) relates a qualitative account of a fifth-grade tutor’s pairing with a third-grade tutee. Raymond, the 
tutor, was pleased to take the lead in his work with Ian, the struggling student. As described, Ian was 
motivated through his work with Raymond. The author describes this as a reciprocal effort in a powerful 
pairing. Unfortunately, it is not clear from the study just how exceptional such a pairing is. In more 
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general terms, Harris, Marchand-Martella, and Martella (2000) report a successful program that relied on 
minimal tutor training. An analysis of specific pairing within this study may have revealed a fair number 
of pairings similar to Raymond and Ian. 

In the previous section, we described the importance of appropriate and substantial tutor training 
with the volunteers. Here, too, we see this need as it applies to even younger tutors. Taylor et al. (1997) 
describe tutor training that includes specific approaches to teaching. When the reader is stuck on a word, 
for example, the tutor should not automatically say the word. In an effort to determine the lack of results 
seen in their study, Jenkins, Jewell, Leicester, Jenkins, and Troutner (1991) suggest that it may have been 
inappropriate instruction that was the cause. Again, we see that there is no strong need for complexity in 
the training and resulting instruction, but it does appear that the training needs to be well suited to the 
tutor, and that the instruction does need to be purposive. And, although having purpose does appear to be 
important, this purpose does not necessarily need to be driven by theory or even curriculum for that 
matter. Tomlin (1995) describes a tutoring program that is just as much a program of mentoring. The high 
school-age tutors were clearly guided by the program, and the program yielded positive academic results. 

Summary of Findings Across Tutoring Subcategories 

A review of the tutoring literature makes it clear that success is achieved through tutoring. 

Elbaum et al. (2000) reports meta-analytic effect sizes of 0.5 standard deviations, Wasik (1998) reviews 
17 programs illustrating varying degrees of success, and Eager (1996) lists a number of positive effects of 
tutoring programs. The Wasik and Slavin (1993) review of studies examined five large-scale tutoring 
models of reading students identified as at-risk for failure in reading skills. The authors report significant 
academic effects of tutoring programs using professional tutors and provide evidence that these effects 
show signs of permanence. 

We have attempted to move beyond these results toward a better understanding of the common 
characteristics of successful tutoring programs. Unfortunately, the insufficient sample sizes and research 
quality found in the tutoring studies leave some important questions unanswered. For example, we would 
like to report whether or not the amount of tutor training correlates with the achievement of their tutees. 
But no research comparison between professional and volunteer tutors was available. We would also like 
to report whether the duration and frequency of the sessions resulted in significant gains. Again, research 
that isolates these variables for comparative purposes was not among the available studies. Further 
research is needed in these areas. 

In the absence of specific supporting research, however, we are able to compile a list of 
characteristics common to the successful tutoring programs described in this chapter. These results are 
presented as a guide to tutoring practitioners in the following section. 
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A Final Word to Practitioners 



The recent studies on tutoring research in this chapter suggest that tutoring can be an effective 
approach in serving at-risk students. The studies presented here are largely studies of early literacy, and it 
is from this body of work that we draw the following preliminary conclusions: 

• Tutors with virtually every level of education have been used, provided the 
intervention was proceded by appropriate tutor training. 

• Effective tutoring sessions are characteristically monitored and adapted with 
appropriate frequency by program implementers. 

• A strong guiding purpose (one that directs tutors in their decision making) seems to 
integral in an effective tutoring program. 

• The diagnostic and prescriptive interaction is encouraged in effective tutoring 
practice. 

The research also suggests the importance of these programmatic considerations: 

• Program logistics such as availability of materials, instructional space, and 
scheduling may have a significant effect on program results. 

• Finding and retaining quality tutors should be a primary concern. 
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In this study of 38 at-risk first-grade readers in the Early Intervention Program (EIP), McCarthy 
et al. report results on a variety of reading and writing measures. The tutored students (n = 19) received 
30 minutes of instruction three times each week for 20 weeks. These sessions are described by the authors 
in detail, details that are similar to the characteristics of the large-scale Reading Recovery program. The 
EIP tutors were reading specialists who were given additional training by the senior researcher. The 
combined result of the tutoring program is reported in an effect size of 0.7 standard deviations. 

Baker, S., Gersten, R., & Keating, T. (2000). When less may be more: A 2-year longitudinal 
evaluation of a volunteer tutoring program requiring minimal training. Reading Research Quarterly, 

55(4), 494-519. 

This Baker et al. article is a study of SMART, a volunteer tutoring program implemented widely 
in Oregon. SMART is an early intervention reading program characterized by its low cost, its brief tutor 
training phase, and the high return rates of program volunteers. The first- and second-grade student 
participants (n = 127) were tutored for two half-hour sessions each week of the school year. Among the 
results of this work is a set of reported effect sizes ranging from .32 to .53 standard deviations on a series 
of five different reading measures. 

Cobb, J. B. (2001). The effects of an early intervention program with preservice teachers as tutors 
on the reading achievement of primary grade at risk children. Reading Horizons, ^7(3), 155-173. 

Manipulatives and games were used in a program described by Cobb in this study. Sixty at-risk 
students were studied as they learned phonological awareness and comprehension strategies by preservice 
teachers. The tutors met with their students for 45 minutes twice each week during the 10-week program. 
Cobb’s presentation of quantitative and qualitative results gives the reader a strong sense of program 
dynamics and success. Among the results, the author reports that the program encouraged significant 
gains in grade one, while the second- and third-grade results were inconclusive. 
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CHAPTEM. 6 



PEER TUTORING 

Zoe Barley 

From the group of studies focused on assisting low-achieving students to meet standards, peer 
tutoring was identified as an intervention that has been studied fairly extensively. Peer tutoring is different 
from tutoring in the sense that peer tutoring involves pairs of students working with each other in the 
learning process. Peer tutoring dates back to the late 18^ century when two British superintendents of 
schools for disadvantaged children adopted peer tutoring in the 1700s as an efficient means to provide 
more instruction to the children. It became popular subsequently in Britain, but the practice waned there, 
and in the U.S., after the late 1800’s. Then in the 1960s, there was a resurgence of interest and use as an 
economical means to increase achievement especially for poor and minority students (Topping, 1988). 
Given the intent of the No Child Left Behind Act of 2001 that all children attain standards, peer tutoring 
again becomes a strategy of importance. Since peer tutoring responds to “an implicit skepticism about 
classroom teachers’ capacity to provide intensive, systematic, effective instruction to a broad range of 
learners via conventional large-group instruction” (Fuchs, Fuchs, Mathes, & Simmons, 1997, p. 180), it 
seeks to provide means for teachers to provide one-on-one attention to low-achieving students. 

Background 

Peer tutoring is “the individualized instruction of one student by another” (Ehly, 1986, p. 2). 

Given the concern of this synthesis with strategies for the regular classroom teacher, for the purposes of 
this chapter peer tutoring is typically as an arrangement of students in the classroom into pairs with the 
pair assisting each other to learn basic facts in a variety of subject areas. Two meta-analyses have been 
completed on peer tutoring, one in 1977 (Hartley) and one in 1982 (Cohen, Kulik, & Kulik). Although 
there were other earlier reviews, Cohen et al. (1982) note limitations with them: “None provided precise 
statements about the size of the learning gains to be expected from tutoring or about the conditions under 
which positive effects are likely to occur” (p. 238). Similarly, a narrative review by Gensemer (2000) 
does not examine the quality of studies nor provide an overall estimate of the effect of peer tutoring. 

Hartley’s (1977) review, a meta-analysis resolving some of the concerns of the earlier reviews, 
focused only on mathematics and had some methodological weaknesses. Cohen et al. (1982) designed 
their work to build on Hartley’s work. More recently, a best evidence synthesis (Mathes & Fuchs, 1994) 
was conducted focusing on peer tutoring as a means of support for children with mild disabilities. The 
best evidence synthesis discusses several other reviews focused on students with disabilities. Since the 
current synthesis is interested in strategies for the regular classroom teacher, these reviews and many of 
the studies they reviewed are less pertinent to this paper. To reiterate, our focus is on low-achieving 
students and includes students with a broad array of handicapping conditions, but the studies included are 
about strategies that occur within the purview of the regular classroom teacher. 
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Researchers describe the value of peer tutoring to be based in its typically teacher-friendly 
materials, comprehensive manuals (for some versions), the fact that it does not intrude on instructional 
time replacing independent seatwork with peer work, and its ability to reach the low-achieving students in 
today’s increasingly diverse classrooms. Although training of teachers to use peer tutoring methods is not 
arduous, typically researchers provide assistance with training of student pairs and ongoing technical 
assistance to ensure that the tutoring is carried out as designed. This support raises untested questions 
about teacher ability to implement peer tutoring in the absence of such support. 

Peer tutoring may involve role reciprocity in which equal-ability students trade roles of tutor and 
tutee during a session. Or an academically stronger student may be paired with an academically weaker 
student. If they are in, or close to, the same grade level, it is peer tutoring; large differences in grade levels 
are more often designated as tutoring. Thus, included in this section of the paper are studies in which 
upper grade children worked with lower grade children if it was a mutual process — that is, if the older 
child also was expected to benefit academically from the tutoring. In some cases, the older child was the 
low-achieving student for which the strategy was intended. But if the older child was academically 
stronger, the study was categorized as tutoring rather than peer tutoring. 

Methodology 



Selection/Exclusion of Studies 

An initial set of 46 studies with peer tutoring referenced was reviewed for inclusion: of these, 30 
are included in this section of the synthesis. Sixteen studies were excluded for the following reasons: (1) 
peer tutoring was part of a complex intervention and could not be examined apart from the full set of 
strategies (n = 7); (2) although called peer tutoring, the strategy did not meet the definition for this review 
(n = 3); or (3) the research was so compromised by design flaws and other biases that the results could not 
be trusted (n = 6). Three of these latter six were single-subject quantitative designs. The comparability of 
the results of single-subject quantitative studies to group designs is not sufficiently clear to include them 
in the review. 

Subgroupings within Peer Tutoring 

The studies on peer tutoring that met the inclusion criteria were then organized into four 
subcategories based on four variations developed by author groups who have conducted the bulk of the 
research in peer tutoring. The most predominant approach in the literature from 1985 to the present is 
Classwide Peer Tutoring (CWPT), developed by Greenwood, Delquadri, and Carta (1988) as part of the 
Juniper Gardens Children’s project at the Schiefelbusch Institute for Life Span Studies at the University 
of Kansas. There are 10 studies, post- 1985, from the group included in this review. Deborah Simmons 
and Douglas and Lynn S. Fuchs and colleagues (Simmons, Fuchs, Fuchs, Hodge, & Mathes, 1994) also 
studied aspects of CWPT including the addition of Peer Assisted Learning Strategies (PALS). There are 
eight studies in this review from their work. H. Edmund Pigott and John W. Fantuzzo and colleagues 
developed Reciprocal Peer Tutoring (RPT) and focused much of their work on mathematics achievement. 
There are six studies included from their work. Finally, six studies, including three from Larry Maheady 
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and colleagues (Maheady & Harper, 1987; Maheady, Sacca, & Harper, 1987, 1988), also have explored 
variations on CWPT. An evaluation of multiple grantees of a federal program completes the set. 

In the rest of this chapter, the four subcategories are discussed one by one with a table including 
brief details for each included study. For each group, there is a description of the studies included, salient 
features of the studies including individual studies of note, and overall conclusions about the group. 
Finally, the paper closes with a summary across groups. 

Classwide Peer Tutoring 

Classwide Peer Tutoring (CWPT) uses a period of time, approximately 30 minutes, in each day or 
for two to five days of the week for pairs of students to work together in learning basic information within 
a subject area. Typically this is time that would otherwise be spent on workbooks or other self-directed 
activities. The teacher must prepare materials for the pairs to use (scripted tutor and tutee interactions that 
include task presentation, question asking, error correction, point earning, feedback, and positive 
reinforcement), assign students to pairs, train the students how to serve as both tutor and tutee, prepare 
assessment materials based on the materials, and monitor the tutoring sessions. In each session, the pairs 
rotate roles, serving as both a tutor and a tutee. In addition, pairs are assigned to teams, and the points are 
awarded to the pairs, according to the mastery of the material by each student, are aggregated into team 
scores. The winning team receives a reward. Usually students are reassigned to new pairs weekly 
(Greenwood, 1991b). 

Multiple studies have been done on CWPT variants. Some are not included in this review because 
they did not include student achievement as an outcome, others because they did not include low- 
achieving students or were not pertinent to regular classroom practice. Some studies specifically looked at 
the relative efficacy of implementation variants (e.g.. Greenwood, Terry, Arreaga-Mayer, & Finney, 

1992; Kohler 8l Greenwood, 1990) and may or may not have included achievement variables. In a recent 
article. Greenwood and associates (2001) describe their 18-year “program of research designed to 
improve literacy of poor culturally and linguistically diverse students and English language learners” (p. 
34). They go on to say: 

Peer-tutoring roles in CWPT are designed to be reciprocal; that is, each student serves as 
both tutor and tutee during each CWPT session. In CWPT, unlike in teacher mediated 
instruction, it is possible for students to experience one-on-one pupil-tutor dyads during 
sessions; relatively immediate error correction; fast pacing and multiple opportunities to 
respond; both teacher and learner roles; written and oral learner response formats; 
inclusion; and social and academic goals addressed during the same instructional time. 

(p. 35) 

Several longitudinal studies and several replications have been completed. Greenwood, Carta, and 
Kamps (1990) summarized the main components that lead to effectiveness as (1) tutoring activities are 
highly structured, (2) teachers carefully monitor tutoring behaviors, (3) tutoring participants receive 
specific training, (4) students are well prepared for the tutoring role, and (5) during tutoring sessions 
teachers monitor the tutoring activity and provide feedback on proper procedures. 
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Overview of Studies 

The 10 post- 1985 studies in this review of CWPT include nine elementary level and one high 
school: seven focused on spelling (three of which also included mathematics and or reading); one on 
reading, mathematics, and language; one on literacy for English language learners; and one on Spanish 
vocabulary (the high school study). The nature of at-risk varies across these studies: two, as indicated in 
Table 6.1, involve language learners; three involve mainstreamed students with mild disabilities; three 
involve comparing students with low socio-economic status (SES) to students with high SES; and two 
involve poor performers identified by standardized tests or low scores on the pretest in the study. 

Preferably studies would have a fairly large sample of students in order to see if there is a 
difference between groups, would use standardized approaches to assessing results, and would compare 
peer tutoring to regular classroom practice. In this case, five studies follow 20 or fewer students in one or 
two classrooms; only one of these has a way to compare to regular teacher practice for low-achieving 
students. Only the two longitudinal studies and the follow-up study used standardized measures with a 
no-treatment comparison group. The rest used weekly teacher-derived tests. Table 6.1 reports the results 
in terms of whether the effect of peer tutoring was either positive for low-performing students in the 
treatment group compared to those in the no-treatment group, even, or negative. When NA is used in the 
“Direction of Result” column, it typically means insufficient information was provided to determine the 
result. 

Findings 

Overall, CWPT is well defined as an intervention, and methods in the studies monitored that it 
was correctly implemented. Tools are available for teacher use (Greenwood, Hou, Delquadri, Terry, & 
Arreaga-Mayer, 2001). Various types of at-risk children have been studied. Several studies are of 
insufficient size or use inadequate measures to add to evidence of the effectiveness of CWPT. Most 
promising as sources of research evidence are the two longitudinal studies and the follow-up study two 
years later. In the case of the 1991 study, the following effect sizes^ were obtained: first-grade reading and 
language .36 and .33 respectively, comparing the treatment group to a control group of similarly low-SES 
students; for a high-SES comparison group with no treatment, the effect sizes compared to control were 
.71 and .48 respectively. Thus, high-performing students also fared well in peer tutoring. The follow-up 
study at sixth grade showed effect sizes of .55, .23, .40, .22, and .49 in reading, mathematics, language, 
social studies, and science. Another way to think of this is that low-SES students who received CWPT 
outperformed their peers who did not earn up to 30 percentage points above them. Even more impressive 
as evidence of the effectiveness of CWPT is the Greenwood 1991a and 1991b study, a longitudinal study 
tracking students from first through fourth grade and comparing low-SES schools in experimental and 
control conditions to students in two high-SES schools in the same district. Despite high losses of 
students due to relocation, and highest in the experimental schools, the effect sizes of the experimental 
group at the end of fourth grade of .57 in reading, .37 in mathematics, and .60 in language are very 



^ The effect size is the difference between the treatment and comparison groups expressed in standardized 
units. An effect size of .3 or greater is considered large enough to be practically meaningful. 
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encouraging. Although the high-SES comparison group had higher effect sizes, there were no statistically 
significant differences in rate of gain; that is, the low-SES group at least held its own. 

Focusing on the six studies with useable results, there is preliminary evidence to support the use 
of CWPT for at-risk elementary children, in particular those from low-SES backgrounds in reading, 
mathematics, and spelling. 



Table 6.1. Post-1985 Studies on Classwide Peer Tutoring (CWPT) 



Author(s) &Year 


Treat 

ment 

Size 


Grade 


Student 

Description^ 


Strategy 

Description 


Subject 

Matter^ 


Type of 
Comparison 


Results^ 


DuPaul, Ervin, Hook, 
& McGoey (1998) 


19 


1-5 


ADHD 


CWPT 


math, 

spelling 


ABAB 2 grp 
post 

nonrandom 


NA 


Greenwood (1991a) 
(longitudinal) 


241 


1,2 


SES 


CWPT 


spelling, 

reading, 

math 


3 grps pre-post 
nonrandom 


All 

positive 


Greenwood, Arreaga- 
Mayer, Utley, and 
Gavin, & Terry (2001) 


117 


1-5 


ELL 


CWPT 


spelling, 

vocabulary 


1 grp pre-post 
nonrandom 


All 

positive 


Greenwood, Delquardi, 
& Hall (1999) 
(longitudinal) 


56 


1^ 


SES 


CWPT 


Reading 

mathematics 

language 


3 grps pre-post 
nonrandom 


All 

positive 


Greenwood, Dinwiddie, 
Bailey, Carta, Dorsey, 
Kohler, et al. (1987) 


174 


1,2 


low test scores 


CWPT 


spelling 


ABAB 1 grp 
post 

nonrandom 


Mostly 

positive 


Greenwood, Terry, 
Utley, Montagna, & 
Walker (follow-up to 
1991) (1993) 


216 


6 


Follow up of 

Greenwood, 

1991 


CWPT 


spelling, 

reading, 

mathematics 


Follow up 


Mostly 

positive 


Madrid, Terry, 
Greenwood, Whaley, & 
Webber 
(1998) 


16 


1 


low test scores 


2 versions of 
PT 


spelling 


ABC 1 grp 

pre-post 

nonrandom 


Mostly 

positive 


Mortsweet, Utley, 
Walker, Dawson, 
Delquadri, Reddy, et al. 
(1999) 


4 


2,3 


mild mental 
retardation 


CWPT 


spelling 


ABAB 1 grp 
nonrandom 


NA 


Sideridis, Utley, 
Greenwood, Delquadri, 
Dawson, Palmer, 
et al. (1997) 


3 


6 


mild 

disabilities 


CWPT 


spelling 


ABAB 1 grp 
nonrandom 


NA 


Wright, Cavanaugh, 
Sainato, & Heward 
(1995) 


16 


High 

School 


LD or poor 
performance 


CWPT 


Spanish 

vocabulary 


1 grp pre-post 
nonrandom 


NA 



^ ADHD= Attention Deficit Hyperactivity Disorder; ELL=English language learner; LD=leaming disabled 
^ ABAB=no treatment, treatment repeated periods; ABC=no treatment, treatment B, then treatment C 
The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 
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Peer-Assisted Learning Strategies (PALS) 



A second group of researchers at Peabody College of Vanderbilt University built on the 
foundation of CWPT. Their work has pursued several different avenues to expand on the earlier research. 
One of their concerns, evidenced in all but one of the studies reviewed, has been the mainstreamed, 
learning-disabled student. Another has been various modifications of CWPT (Simmons et al., 1994) as 
well as how the student dyads are assigned and how teachers are supported in using CWPT. The 1994 
Mathes, Fuchs, Fuchs, Henley, and Sanders article provides an overview of the method. 

Within this group of researchers are those who developed the Peabody PALS (Peer Assisted 
Literacy Strategies) for lower elementary grades. This intervention expanded the original CWPT to 
engage students in strategic reading activities (Mathes, Howard, Allen, & Fuchs, 1998) designed to 
increase student abilities to (1) cumulatively review information read, (2) sequence information, (3) 
sununarize paragraphs and issues, (4) state main ideas in as few words as possible, and (5) predict and 
check outcomes. PALS versions are described as subject specific and grade specific and include content 
materials for teacher and student use (e.g.. First Grade PALS was developed as an earlier grade version of 
Peabody PALS). Students are paired within the classroom and complete a tutoring routine. (In several 
studies, a stronger student assumed the tutor role and a low-achieving student was assigned the tutee role; 
that is, the tutoring was not reciprocal.) Pairs earn points and are assigned to teams such that a winning 
team can be announced at the end of the week. The incorporation of critical content identified through 
research sets the literacy PALS apart. 

Overview of Studies 

Within this author group’s work, eight studies were found that met the inclusion criteria (see 
Table 6.2). Each of them examined peer tutoring with elementary age children, and all but one focused on 
learning disabled children in the regular classroom. The remaining one identified low-achieving students 
through performance on a series of screening tests. The Simmons, Fuchs, Fuchs, Mathes and Hodge study 
(1995) examined the use of Explicit Teaching (ET), a form of direct instruction, given the general 
thinking that it is a better methodology for the lower performing student. In the most recent study looking 
at PALS and elementary mathematics, kindergarten students as reciprocal peers were studied. 

The studies in this group are of high quality with careful attention to fidelity of implementation, 
teacher satisfaction, training of teachers, and increasingly more rigorous use of measurements and 
analyses. Limitations generated in part by the implementation methods used to ensure rigor are that the 
support for teachers (e.g., assistants often train the students in CWPT methods) and the observations and 
consultation for teachers by the researchers during implementation may compromise generalizability. 
Most peer tutoring studies are coded as nonrandom in the sense that teachers are assigned to experimental 
or control groups but not students whose scores are the unit of analysis. 

Findings 

To a degree, the variety in the studies of grade levels, subject matter, and aspects of peer tutoring 
adds complication to determining what we now know or do not know about the use of peer tutoring in 
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elementary classrooms. One of the more recent studies is a replication (Mathes & Babyak, 2001) in which 
the use of teacher-directed advanced mini-lessons for the low-achieving students was added to give them 
a boost as they engaged in peer tutoring. This study also did not use reciprocal tutoring but paired a 
stronger student with a weaker. Despite some problems with teacher implementation of the mini-lessons, 
effect sizes of 1.22, 1.14, 1.37, and .50 on word identification, word attack, basic skills, and passage 
comprehension for low-average students offer great promise for use with low-achieving early readers. 

The most recent mathematics study with kindergarten children (Fuchs, Fuchs, & Kams, 2001) produced 
mixed and somewhat difficult to interpret results on its two outcome measures but does demonstrate that 
kindergarten students can engage in peer tutoring. Effect sizes of .46 and .41 for low-achieving and 
disability students on a math readiness measure, and .3 1 and .22 on a primary mathematics measure, also 
show promise for peer tutoring in the early grades. 

This set of studies provides preliminary evidence on the effectiveness of peer tutoring with 
elementary level children who are learning disabled in math and reading. 



Table 6.2. Post-1985 Studies on Peer- Assisted Learning Strategies (PALS) 



Author(s) &Year 


Treat 

ment 

Size 


Grade 


Student 

Description" 


Strategy 

Description 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Fuchs, Fuchs, 
Hamlett, Phillips, 
& Bentz (1994) 


30,30 


2-5 


LD, LP as 
determined by 
testing 


PT + weekly 
feedback to 
teachers, 
feedback 
only 


Math 


3 grps pre-post 
nonrandom 


Mostly 

positive 


Fuchs, Fuchs, & 
Kams (2001) 


84 


K 


Spec ed + low 
test scores 


PALS 


Math 


2 grps pre-post 
nonrandom 


All 

positive 


Fuchs, Fuchs, 
Mathes, & 
Simmons (1997) 


60 


2-6 


LD & low test 
scores 


PALS 


reading 


2 grps pre-post 
nonrandom 


All 

positive 


Fuchs, Fuchs, 
Phillips, Hamlett, 
& Kams (1995) 


120 


2-5 


LD & low test 
scores 


PALS 


math 


2 grps pre-post 
nonrandom 


Mostly 

positive 


Mathes & Babyak 
(2001) 


130 


1 


Teacher 
determined LP 
+ low test 
screening 


PALS, 
PALS + 
mini lessons 


reading 


3 grps pre-post 
nonrandom 


All 

positive 


Mathes, Howard, 
Allen, & Fuchs 
(1998) 


96 


1 


Low screening 
test 


PALS 


reading 


2 grps pre-post 
nonrandom 


All 

positive 


Simmons, Fuchs, 
Fuchs, Hodge, & 
Mathes ( 1 994) 


118 


2-5 


LD + teacher 
determined LP 


4 

modification 
s of CWPT 


reading 


4 grps pre-post 
nonrandom 


Mostly 

positive 


Simmons, Fuchs, 
Fuchs, Mathes, & 
Hodge (1995) 


68 


2-5 


LD + teacher 
determined LD 


Explicit 
teaching 
(ET), ET 
with PT 


reading 


2 grps pre-post 
nonrandom 


Mostly 

positive 



“ LD=leaming disabled; LP=low performing 

^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 
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Reciprocal Peer Tutoring in Elementary Mathematics 



Fantuzzo, Davis, and Ginsburg (1995) describe the Reciprocal Peer Tutoring (RPT) strategy as 

follows: 



The RPT strategy is designed to enhance learner control and peer cooperation. It provides 
a pair of students with the opportunity to alternate between the student and teacher roles. 

The intervention includes a structured peer-tutoring format to guide the dyad through the 
learning process and a cooperative reward system that the students help design, evaluate, 
and use to manage their own reinforcement procedures, (p. 273) 

Ginsburg-Block and Fantuzzo (1997) add, “RPT emphasizes having the peer teacher prompt and 
praise the student and seek the classroom teacher’s help if needed, rather than having the tutor provide 
instruction or elaborate explanations beyond their ability level” (p. 136). 

Overview of Studies 

In the earliest study of this post- 1985 group (see Table 6.3) Pigott, Fantuzzo and Clement (1986) 
trace the development of RPT to literature supporting an intervention that “combines the benefits of peer 
tutoring, interdependent group reward contingencies, and individual accountability procedures” (p. 93). 
The 1986 study is a replication and expansion of an earlier Pigott study. This study does not use pairs of 
students, typically what is meant by peer tutoring, but assigns groups of four classmates to four tutoring 
roles. The three groups of low-achieving students increased their performance to a level indistinguishable 
from their higher performing peers. 

There are five additional studies in this group. Following the Pigott et al. study (1986), Fantuzzo, 
Polite, and Grayson (1992) found that the combination of two components — the interdependence of the 
members of the pair in achieving their goals and student pair choice in selecting rewards — was most 
effective in increasing mathematics computation rates. He called the two elements the structure of the 
RPT (structured so the members of the pair are interdependent) and the reward (that the pairs receive 
rewards of their own choosing). When both elements were present, the low-performing pairs 
outperformed the comparison groups with one or none of the elements. Across all groups, choosing the 
reward resulted in higher performance, but the same was not true for the structure of interdependency. 
Heller and Fantuzzo (1993) added a parent involvement intervention and compared RPT, with and 
without parent involvement, to a control group. The 1995 Fantuzzo et al. study, an extension of the Heller 
and Fantuzzo study, separated out the parent involvement component for individual study. Ginsburg- 
Block and Fantuzzo (1997) built on a Fuchs, Fuchs, Hamlett, Phillips, and Bentz (1994) study that 
compared peer tutors trained in the process to those not trained. This study added observations of the 
actions of the peer tutors and an achievement outcome measure — math computations — as well. 

Findings 

Increased achievement for low-achieving students was evidenced with the use of RPT in each of 
these studies (except the Pigott study and the 1995 Fantuzzo study, which only looked at RPT combined 
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with parent involvement). Three studies explored mediating factors of the intervention, and two looked at 
added parent involvement elements. These studies were able to randomly assign students by selecting 
them from their regular classes and placing them in an experimental classroom. This classroom was 
managed as a classroom teacher would but conducted by the research team. What has not been examined 
here is the management of RPT by typical classroom teachers. Teachers were actively involved; whether 
there would be difficulty in managing RPT as part of the regular classroom experience has not been 
demonstrated. Adding the parent involvement, although a strong component, adds a management burden 
for the teacher. 

Given that these studies used random assignment of children to groups, the six studies provide 
conclusive evidence for the effectiveness of peer tutoring for elementary children identified as low 
achieving in math. However, whether these findings can be generalized to the regular classroom has not 
been demonstrated. 



Table 6.3. Post-1985 Studies on Reciprocal Peer Tutoring (RPT) 



Author(s) 4&Year 


Treat 

ment 

Size 


Grade 


Student 

Description^ 


Strategy 

Description 


Subject 

Matter 


Type of 
Comparison 


Results^ 


Fantuzzo, Davis, & 
Ginsburg (1995) 


26 


4-5 


LP in math, 
Afr Amer 


RPT+ parent 
involvement, 
parent 
involvement 


math 


3 grps pre- 
post random 


All 

positive 


Fantuzzo, King, & 
Heller (1992) 


16 


4-5 


LP, Afr Amer 


Full RPT, reward 
only, structure 
only 


math 


4 grps pre- 
post random 


All 

positive 


Fantuzzo, Polite, & 
Grayson (1990) 


12 


4-5 


LP, low SES 


RPT 


math 


2 grps post 
random 


All 

positive 


Ginsburg-Block & 
Fantuzzo (1997) 


20 


4-5 


LP, low SES 


RPT 


math 


2 grps pre- 
post random 


All 

positive 


Heller & Fantuzzo 
(1993) 


28 


4-5 


LP, Afr Amer 


RPT, RPT + 

parent 

involvement 


math 


3 grps pre- 
post random 


All 

positive 


Pigott, Fantuzzo, & 
Clement (1986) 


12 


5 


LP 


RPT - 4 member 
group 


math 


2 grps 
ABAB pre- 
post random 


NA 



“ LP=low performing 

^^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 



Additional Peer Tutoring Studies 

Six additional peer tutoring studies were found (see Table 6.4) — four of which are further 
examinations of Classwide Peer Tutoring (CWPT); one of which (Maheady et al., 1988) is a further 
development of CWPT called CSTT; and a sixth, which is simply designated as peer tutoring. One of the 
studies examines peer tutoring at the middle school level in math, and three examine peer tutoring at the 
high school level in math, social studies, and history. Four of the six studies use ABAB single-group 
reversal designs with comparisons made between periods of treatment and baseline or periods of no 
treatment. Several upper-grade-level studies note that the use of peer tutoring has been applied only to 
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basic skills and that much remains to be understood about applications to higher order thinking skills. One 
study (Allsopp, 1997) examining eighth-grade student achievement in algebra did develop more complex 
worksheets for the tutor pairs to use. This study, which had a nonrandom comparison group using class as 
the unit of analysis, found no effect. 



Table 6.4. Post-1985 Studies on Various Forms of Peer Tutoring 



Author(s) 

Year 


Treat 

ment 

Size 


Grade 


Student 

Description^ 


Strategy 

Description 


Subject 

Matter 


Type of 
Comparison 

b 


Results*^ 


Allsopp (1997) 


140 


8th 


Low- 

achieving in 
math 


CWPT 


algebra 


2 grps pre- 
post non- 
random 


Even 


Bell, Young, Blair, 
& Nelson (1990) 


59 


9,10 


LD in 

diverse 

classroom 


CWPT 


history 


1 grp pre- 
post 


NA 


Cardona & Artiles 
(1998) 


19 


1st 


ESL in 
diverse 
classrooms 


PT and 
mixed ability 
grps 


math 


1 grp pre- 
post 

nonrandom 

ABAB 


Mostly 

positive 


Maheady & Harper 
(1987) 


70 


3,4 


Low- 

achieving in 
spelling 


CWPT 


spelling 


1 grp pre- 
post 

nonrandom 

ABAB 


All 

positive 


Maheady, Sacca, & 
Harper (1987) 


91 


9,10 


Mild 

handicapped 
in diverse 
classrooms 


CSTT - 3 
person teams 


math 


1 grp pre- 
post 

nonrandom 

ABAB 


NA 


Maheady, Sacca, & 
Harper (1988) 


50 


10th 


Mild 

handicapped 
in diverse 
classrooms 


CWPT 


Social 

studies 


1 grp pfe- 
post 

nonrandom 

ABAB 


NA ■ 



“ LD=leaming disabled; ESL=English as a second language 
^ ABAB=no treatment, then treatment periods repeated. 

^The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative, from 
Cooper, Charlton, Valentine, and Muhlenbruck (2000). 

Finally, one additional study (Pringle, Anderson, Rubenstein, & Russo, 1993), an evaluation of 
the fundees of a large federal program, not included in the table, warrants comment. The program funded 
31 one-year grants to explore innovative ways of helping disadvantaged secondary students attain 
proficiency in basic and more advanced skills. Data from 13 grantees were used in the report. Findings 
suggest that peer tutoring and mentoring can be useful in addressing the needs of disadvantaged 
secondary students. Eight programs showed modest positive effects on academic achievement. However, 
the wide variance in both implementation factors and outcome measures limits the useable data from the 
programs. And, to a large extent, the tutoring seems to have occurred outside the regular classroom. 



Summary of Findings Across Studies of Peer Tutoring 

Reviewing the studies of CWPT both in the Greenwood group and in the final group of studies, 
we find that the moderate to higher quality studies provide evidence of effectiveness of CWPT for low- 
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SES and low-achieving elementary students in spelling, reading, and math. There are well-developed 
materials to support teacher use. Across all studies of peer tutoring, a variety of types of at-risk students 
have been studied. With the exception of Allsopp’s study (1997) and the studies on the use of the PALS 
materials, the focus has been understandably on basic skills. Researchers express some concern about the 
amount of training required, of both teachers and students, and more important, the classroom 
management logistics of tutoring materials including weekly quizzes, assignment and reassignment to 
peer pairs, score-keeping processes, and rewards. 

The Fuchs, Fuchs, and Mathes PALS studies move the use of peer tutoring beyond basic skills. 
The development of materials therefore requires more than assembling lists of spelling words or basic 
arithmetic computations. Some PALS materials are available for teacher use. Results are positive, with 
large effect sizes for PALS plus teacher-provided advanced mini-lessons on new material. 

The six Fantuzzo and associates studies also found positive effects and, with random assignment 
of students to groups, were able to rule out alternative explanations for the results. Their emphasis on 
student peer pairs being involved in goals setting and selecting rewards from a menu of choices is an 
enhancement of peer tutoring that has evidence of effectiveness. These studies provide conclusive 
evidence for the use of RPT with low-performing elementary students in math. 

A Final Word to Practitioners 

Overall, there is evidence that peer tutoring, when well developed and well supported, can 
improve achievement for low-achieving students. For CWPT, there is a body of recent evidence 
supporting its application with low-SES elementary children in reading, mathematics, and spelling. The 
body of evidence supporting the use of PALS extends this to elementary students classified as learning 
disabled. And the Fantuzzo and associates’ work with variations on reciprocal peer tutoring further 
confirms the efficacy of forms of peer tutoring for low-achieving elementary students in learning basic 
material in math. 

The main components of PT that lead to effectiveness are as follows: 

• tutoring activities are highly structured, 

• teachers carefully monitor tutoring behaviors, 

• tutoring participants receive specific training, 

• students are well prepared for the tutoring role, and 

• during tutoring sessions, teachers monitor the tutoring activity and provide feedback 
on proper procedures. 

Thus, to use peer tutoring in a setting, materials must be developed for weekly use, including 
tutoring materials such as flash cards and assessment materials. The time and process for the tutoring 
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sessions must be established and pairing arrangements planned. Teachers will need training and logistical 
support to initiate peer tutoring. 

The following references are good resources for examining the requirements and deciding 
whether and how to conduct peer tutoring: 

Fuchs, D., Fuchs, L. S., & Burish, P. (2000). Peer-assisted learning strategies: An evidence-based 
practice to promote reading achievement Learning Disabilities Research & Practice, 75(2), 85-91. 

Greenwood, C. R., Carta, J. J., & Kamps, D. (1990). Teacher-mediated versus peer-mediated 
instruction: A review of advantages and disadvantages. In H. C. Foot, M. J. Morgan, & R. H. Shute 
(Eds.), Children helping children (pp. 177-206). Chichester, England: Wiley. 

Greenwood, C. R., Carta, J. J., & Hall, R. V. (1988). The use of peer tutoring strategies in 
classroom management and educational instruction. School Psychology Review, 77(2), 258-275. 

Greenwood, C. R., Hou, L. S., Delquadri, J., Terry, B., & Arreaga-Mayer, C. (2001). The Class 
Wide Peer Tutoring Learning Management System (CWPT-LMS). In J. Woodward & L. Cuban, 
Technology, curriculum, and professional development: Adapting schools to meet the needs of students 
with disabilities. Newberry Park, CA: Corwin. 

Topping, K. (1988). The peer tutoring handbook: Promoting cooperative learning. Cambridge, 
MA: Brookline Press. 
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Annotated Bibliography of Selected References 

The studies selected for the annotated bibliography are representative of the four variations of 

peer tutoring found in the search of the literature plus an evaluation of federally fimded programs. 
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Fuchs, L. S., Fuchs, D., & Karas, K. (2001). Enhancing kindergartners’ mathematical 
development: Effects of peer-assisted learning strategies. Elementary School Journal 707(5), 495-510. 

This study examined whether kindergartners could be effective as peer tutors in twice weekly 
sessions in mathematics. Kindergarten students were taught how to alternately serve as tutor and tutee in 
paired sessions. Tutoring sessions were monitored to ensure that tutoring occurred as intended. Students 
at all achievement levels benefited. Teachers judged the treatments to be feasible and effective. 

Ginsburg-Block, M., & Fantuzzo, J. (1997). Reciprocal peer tutoring: An analysis of teacher and 
student interactions as a function of training and experience. School Psychology Quarterly, 72(2), 
134-149. 

This study examined whether reciprocal peer tutoring for at-risk, fourth- and fifth-grade students 
in mathematics would improve achievement. Students were randomly assigned to peer tutoring or a 
control experience in which students were not paired. Those in peer tutoring conducted two 45 -minute 
sessions each week for 10 weeks. Peer tutors were carefully trained in the roles. Those in peer tutoring 
outperformed the control group and also demonstrated higher rates of on-task behaviors. 

Greenwood, C. R., Delquardi, J. C., & Hall, V. (1989). Longitudinal effects of classwide peer 
tutoring. Journal of Educational Psychology, 57(3), 371-83. 

The researchers conducted a longitudinal study of the effects of Classwide Peer Tutoring (CWPT) 
on low-SES elementary school students’ achievement. Classwide peer Tutoring involves pairs of students 
alternately assuming the role of tutor and tutee. A low-SES control group received traditional classroom 
instruction, as did a high-SES comparison group. CWPT was successively introduced in three subject 
areas: spelling, mathematics, and reading culminating in a planned use of 90 minutes per day. The low- 
SES treatment group outperformed the low-SES control group, as did the high-SES comparison group. 
Attrition was a problem over the four years of the study with the low-SS group losing the largest number 
of students. 

Mathes, P. G., & Babyak, A. E. (2001). The effects of literacy strategies for first-grade readers 
with and without additional mini-skills lessons. Learning Disabilities: Research & Practice, 7d(l), 

28-44. 



This study sought to examine whether first-grade, Peer-Assisted Literacy Strategies (PALS) 
would be effective for children of different achievement levels in naturally constituted regular classrooms. 
Children were paired to practice aspects of phonics and share reading and reflection on text for 30 
minutes three times a week. As an additional support, teachers provided mini-lessons on the same 
material. First-grade PALS was effective in enhancing reading performance of low-achieving students. 
The addition of mini-lessons enhanced the results but teachers resisted implementing them. 

Pringle, B., Anderson, L. M., Rubenstein, M. C., & Russo, A. W. W. (1993). Peer tutoring and 
mentoring services for disadvantaged secondary school students: An evaluation of the Secondary Schools 
Basic Skills Demonstration Assistance Program. Washington, DC: ERIC. 
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This report is an evaluation of the federally funded Secondary Schools Basic Skills 
Demonstration Assistance Program. The program funded 31 one-year grants to explore innovative ways 
of helping disadvantaged secondary students attain proficiency in basic and more advanced skills. Data 
from 13 grantees were used in the report. Findings suggest that peer tutoring and mentoring can be useful 
in addressing the needs of disadvantaged secondary students. Eight programs showed modest positive 
effects on academic achievement. However, the wide variance in both implementation factors and 
outcome measures limits the useable data from the programs. 
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CHAPTEM. 7 



COMPUTER-ASSISTED INSTRUCTION 

Motoko Akiba 

Computer-assisted instruction (CAI) has been identified as an effective strategy to improve the 
achievement of at-risk students. With the advancement of technology, the use of computers in schools has 
rapidly increased over the last 20 years. Statistics show that by 1996, 70 percent of fourth graders and 50 
percent of eighth and eleventh graders were using a computer at school at least once a week, while less 
than 20 percent did so 20 years ago (U.S. Department of Education, 2002). Accordingly, the use of 
technology to improve student learning has become one of the major emphases in the current education 
reform as expressed in the No Child Left Behind Act of 2001. 

Researchers have argued that CAI has the potential to alter the nature of teaching fi-om the 
traditional, teacher-centered model to a more student-centered instruction approach which especially 
benefits students at risk (Waxman & Huang, 1996; Waxman, Padron, & Arnold, 2001). Given the current 
reform focus on low achievers in high-poverty schools and the promise of CAI to improve the 
achievement of this population, synthesizing the effectiveness of CAI based on the available empirical 
studies can provide important information for policymakers and educators. 

This chapter presents the results from a synthesis of 21 empirical studies published since 1985 
that examine the effects of CAI on the achievement of at-risk students. The concept of computer-assisted 
instruction is defined, the methods used to conduct the meta-analysis are described, the results of the 
analysis are reported, and a discussion of the findings from selected studies that examined the special 
features of CAI is provided. Finally, policy implications and suggestions for future studies on the effects 
of CAI for at-risk students are presented. 



Background 

Benefits of CAI in terms of the achievement of at-risk students have been discussed in many 
studies (e.g., Waxman et al., 2001; Hessemer, 1986; Cantrell, 1993). These researchers suggest that the 
following characteristics of CAI contribute to the learning of at-risk students: 

• CAI is non-judgmental and motivational. 

• CAI gives frequent and immediate feedback. 

• CAI can individualize learning through designs to meet students’ needs. 

• CAI allows for more student autonomy. 

• CAI provides a multi-sensory learning environment (images, sounds, and symbols). 
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Research suggests that teachers in high-poverty schools tend to use teacher-dominated whole- 
class instruction approaches more often than do teachers in middle-class schools (Devillar & Faltis, 

1991). Researchers have observed that CAI changes teacher-dominated classroom interactions to student- 
centered ones (Swan & Mitrani, 1993, Waxman & Huang, 1996), and that it enables teachers to 
individualize instruction (Sandholtz, Rnagstaff, & Dwyer, 1992). In addition, CAI motivates students to 
be actively engaged in learning tasks (Worthen, Vanduzen, & Sailor, 1994). Furthermore, CAI may be 
especially effective for kinesthetic learners because of its capacity to provide multi-sensory learning 
environments. It can connect student classroom learning to real-life situations through the use of images, 
sounds, and symbols (Kozma & Croninger, 1992; Means & Olson, 1994; Poirot & Canales, 1993). 

Although many studies have examined the effects of CAI on the performance of low-achieving 
students, there is no recently published meta-analysis of the effects of CAI on this population. The two 
syntheses that have examined the effects of CAI on beginning readers included both low and high 
achievers (Blok, Oostdam, Otter, & Overmaat, 2002) and students with learning disabilities (Hall, 

Hughes, & Filbert, 2000). This chapter provides additional information for policymakers and educators by 
systematically summarizing CAI effects on at-risk students through a meta-analysis. 

Computer-assisted instruction (CAI) is generally defined as a process in which a computer is 
used to present instructional material, monitor the progress of learning, and select additional teaching 
materials in view of a learners’ present level of performance (Kestner, 1989; Hessemer, 1986). This 
synthesis considers only CAI that is designed to assist K-12 low achievers or students at risk, during the 
regular school day. CAI programs implemented with primarily special education students and high 
achievers were not examined. In the studies reviewed, students worked individually or in pairs/groups 
using a computer program that guided their learning. The teacher’s role varied depending on the nature of 
the computer software — some software programs required teachers’ input to facilitate the instruction, 
whereas others required only a minimal role such as troubleshooting. 

Methodology 

Selection/Exclusion of Studies 

Using the selection criteria described in Chapter I, a total of 25 studies were examined. Of these, 
three studies were excluded because the effects of CAI could not be separated from the effects of the 
larger program that included CAI, and one study was excluded because it dealt with computer-assisted 
testing, rather than instruction. All of the remaining 2 1 studies employed quantitative methods to examine 
the effects of CAI. Of these, 17 studies provided enough statistical information to compute effect sizes, 
making it possible to conduct a meta-analysis. The four studies that did not provide sufficient information 
(Francis, 1990; Koza, 1989; Swarm, 1991; Tanner, 1987) were excluded from the analysis. 

Meta-analysis 

Meta-analysis is a research method that examines the results of a number of studies in order to 
determine the average effect of a given intervention and identify moderating. Only studies that utilize 
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quantitative methods can be included in a meta-analysis factors (see Chapter 1 and Appendix B for 
details). 



First, effect sizes were computed for each study, and then an overall effect size across the 17 
studies was computed. Then, moderating factors were examined in order to understand how they interact 



with interventions to influence the effect size observed. For example, subject area is one such factor, and 
the effect size of CAI use in mathematics may be larger than the CAI use in literacy. 

Based on past research and on the information available from the sample of studies, we examined 
research quality and three program characteristics: subject area (e.g. math, literacy), grade level, and the 
nature of computer-assisted instruction. 

It is important to understand whether and how effect sizes of CAI are influenced by the quality of 
the study. If effect sizes differ as a function of the study quality, it is wise to pay more attention to the 
effect sizes presented in high-quality studies. 

The association between the effect of CAI and the program characteristics of subject area, grade 
level, and the nature of CAI is important information for policymakers and practitioners. For example, if 
CAI in mathematics is more effective than in literacy, and if CAI is more effective for elementary school 
students than for middle school students, policymakers and practitioners might first implement CAI in 
mathematics in elementary schools. 

Researchers have debated the differences between problem-solving-oriented CAI and skill- 
oriented CAI in improving the achievement of at-risk students (Emihovich & Miller, 1988; Wepner, 

1991). Whether the effectiveness of CAI differs depending on the nature of CAI and, if so, how it differs 
are other important issues for policymakers to consider. 



Description of the Selected Studies 

Table 7. 1 describes the characteristics of the 17 studies selected for the meta-analysis. The 
publication year ranged from 1986 to 2002. There were only a few recent studies that met the selection 
criteria. The table shows that all of the studies examined at-risk students, though these students were 
defined differently across the studies. The grade level of students involved in the studies ranged from 1st 
grade to 1 2th grade. Most studies employed pretest and posttest comparisons between treatment and 
control/comparison groups. Two studies conducted only posttest comparisons, and two other studies 
reported pretest and posttest differences without control or comparison group. Outcome measures were 
mathematics test scores and literacy test scores that included reading, vocabulary, writing, listening, and 
language. The last column in the table indicates the direction of the study results. 



Meta-Analysis Results 
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Table 7.1. Characteristics of the Studies Examining the Effects of Computer-Assisted Instruction 
on the Achievement of At-Risk Students 



Author(s) &Year 


Treat 

ment 

Size 


Grade 


Student 

Description 


Type of 
Comparison 


Subject 

Matter 


Results” 


Adams (1986) 


45 


7,8 


Below grade level 


2 grps. pre- 
post random 


reading 


Even 


Bailey (1991) 


21 


9 


Between 1®^ and 30^ 
national percentile 


2 grps. pre- 
post random 


math 


All 

positive 


Dellario(1987) 


95 


9 


Enrolled in a basic skill 
curriculum 


2 grps. pre- 
post matched 


reading, 

math 


All 

positive 


Dungan (1990) 


121 


4-6 


Below the 50*^ percentile 


2 grps. pre- 
post matched 


reading, 

math 


Even 


Emihovich & Miller 
(1988) 


4 


1 


Low-achieving African 
American students 


3 grps. post 
random 


math, 

vocabulary 

language 


Even 


K.estner(1989) 


60 


7,8 


Chapter 1 remedial math 
program students 


3 grps. pre- 
post matched 


math 


All 

positive 


K.itabchi(1987) 


23 


5 


African American students 
in an inner-city school 


3 grps pre-post 
non-random 


reading, 

math 


All 

positive 


Kochinski (1986) 


20 


10-12 
yr olds 


One year or more below 
grade level 


2 grps. pre- 
post random 


reading 


Even 


Ligas (2002) 


99 


6-8 


At-risk students 


2 grps. Pre- 
post non- 
random 


reading 


All 

positive 


Mickens (1991) 


50 


9-12 


At-risk minority students 


2 grps. pre- 
post random 


Math 


All 

positive 


Moore (1988) 


61 


7,8 


Students in remedial math 
classes 


4 grps. pre- 
post non- 
random 


math 


All 

positive 


Ramey (1991) 


138 


. 2-5 


Given compensatory 
education 


2 grps. pre- 
post non- 
random 


reading, 

math 


Even 


Sinkis (1993) 


824 


2-6 


Chapter 1 students 


2 grps. pre- 
post non- 
random 


reading, 

math 


All 

positive 


Stegemann (1986) 


47 


4 


Under-achievers 


3 grps. pre- 
post random 


math 


Even 


Weller, Carpenter, & 
Holmes (1998) 


63 


5 


Low-achievers 


2 grps. pre- 
post random 


reading, 

math 


Mostly 

positive 


Wepner (1991) 


34 


8 


At-risk students 


2 grps. post 
random 


reading, 

writing 


Even 


Willingsky & Green 
(1990) 


19 


7-9 


Potential candidates for 
remedial programs 


3 grps. pre- 
post non- 
random 


reading, 

writing 


Even 



^ The number of comparisons that were all positive, mostly positive, even, mostly negative, or all negative from 
Cooper , Charlton, Valentine, and Muhlenbruck (2000). 



Overall Effect Size 

Effect sizes for the 26 independent samples from 17 studies were computed. The effect sizes 
varied from -.12 to 1.36. The overall effect size was .37, which can be translated into a percentile 
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difference of 14 points between the average achievement of the treatment group and the control group. In 
other words, the average students who received CAI scored 14 percentile points higher than the average 
students who did not receive CAI. It was found that the effect size could vary from .30 to .44. Because the 



effect size is greater than zero, it is assumed to have a significantly positive effect. These results 
demonstrate that CAI is an effective strategy that significantly increases the achievement of at-risk 
students. 

Moderators of CAI Effects 

The influence of research quality and three program characteristics of grade level, subject area, 
and the nature of CAI was examined. The sample size, the presence of pretest and control groups, 
sampling methods (random or non-random), alignment between research questions and study design, 
program implementer qualifications, and the reliability of the measurement of key variables were 
considered in determining the quality of studies. Study quality was coded as low or high. 

The subject areas in which computer-assisted instruction was conducted were mathematics, 
reading/vocabulary, writing, listening, and language arts. Due to the small number of studies that reported 
results relative to writing, listening, and language arts, these areas were combined with 
reading/vocabulary to create an inclusive category, “literacy.” Mean effect sizes were compared between 
the CAI programs on math and literacy. 

Grade levels were coded as lower elementary (first grade and second grade), upper elementary 
(third grade to fifth grade), middle school (sixth grade to eighth grade), and high school (ninth grade to 
12th grade). When the samples’ grade levels spanned multiple grade bands, the mean grade level was 
used to determine the coding. 

The nature of CAI was determined by reviewing the descriptions of the CAI programs in terms of 
whether they focus on drill and practice, project and problem solving, or both. 

J-test statistics were used to analyze the effects of moderating factors with two categories, and F- 
test statistics were used to analyze the effects of moderating factors with three or more categories. The 
mean effect sizes for high- and low-quality studies did not significantly differ. The results for program 
characteristics are presented in Table 7.2. 

Regarding the program characteristics, 48 out of 61 achievement outcomes involved CAI effects 
on students in upper elementary grades (3-5) or middle school grades (6-8). Nine achievement outcomes 
involved CAI effects on lower elementary grade students (1-2), and only four involved CAI effects on 
high school students (9-12). The effect sizes by grade level varied from .20 to .62, with the highest mean 
effect on high school students and the lowest mean effect on middle school students. The mean CAI effect 
size for lower elementary grade students (.59) was similar to that for high school students (.62), followed 
by the mean effect on upper elementary grade students (.31). Due to the small sample size and large 
standard deviation, the differences between these means were not statistically significant, but the 
numerical mean differences by grade level may merit attention. 
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Table 7.2. Moderators of Effect Sizes: Computer- Assisted Instruction 



Program Characteristic 
(N = 61)“ 


Categories of 
Characteristic 




Mean (SD) 


t/F 


Probability 


Grade Level 


Lower elementary (1-2) 


9 


.59 (.56) 


F=\l'i 


.302 


Upper elementary (3-5) 


27 


.31 (.70) 






Middle school (6-8) 


21 


.20 (.46) 






High school (9-12) 


4 


.62 (.28) 






Subject Area 


Mathematics 


25 


.57 (.74) 


t=2.7^ 


.007 


Literacy 


35 


.16 (.40) 






Type of CAI 


Drill/practice 


19 


.13 (.47) 


F=1.53 


.229 


Project/problem solving 


25 


.32 (.48) 






Mixed 


4 


.51 (.15) 







“N is the total number of achievement outcomes across all studies. 
^The analysis was conducted treating achievement outcomes as a unit. 



Twenty-five comparisons were made to examine the effects of CAI in mathematics, and 35 
comparisons were made to examine the effects of CAI in literacy. One study combined the effects of CAI 
in mathematics and literacy, preventing the examination of CAI effects separately. The mean effect size 
for CAI in mathematics was .57, as compared to a mean effect size of .16 in literacy. The difference 
between these effect sizes was statistically significant. Based on these results, CAI in math improves 
achievement among at-risk students more effectively than does CAI in literacy. 

Forty-eight of 61 comparisons were accompanied with explanations on the type of the CAI 
implemented. Nineteen comparisons examined the effects of CAI focused on drill and practice; 25 
comparisons examined the effects of CAI focused on project and problem solving. Four comparisons on 
CAI programs included both drill/practice and project/problem solving elements. The mean effect size 
was the largest for the mixed program (.51), followed by project/problem solving (.32) and drill/practice 
(.13). However, these differences were not statistically significant. 

The Effects of Special Characteristics Related to CAI 

The Role of Teachers 

Most of the studies did not provide information on how the teacher was involved in CAI. Given 
the fact that the role of teachers mediating between students and computers is a crucial influence on 
students’ learning experiences, it is worth focusing on the findings from the few studies that examined the 
role of teachers involved in CAI. 

Kestner (1989) compared the effects of CAI with two different conditions: (1) computer-managed 
components of instruction that involves teacher decisions about what software is presented to students, 
and (2) a computer management system that diagnoses and prescribes software for individual students. 
Kestner found that the group of seventh and eighth graders who had teacher input on the assignments of 




software topics showed higher achievement gains in mathematics than did the group of seventh and 
eighth graders without teacher input. 

Moore (1988) examined the interaction between teachers’ personalities and CAI on the 
achievement of seventh- and eighth-grade remedial math students. Teachers’ personalities were rated as 
having positive affect or negative affect based on self-nomination and through interviews with their 
colleagues. CAI was found most effective when taught by teachers who had generally positive affect — 
those who frequently praised students and had high expectations for their students. 

The role of tutors may also influence student learning through CAI in ways similar to the role of 
teachers. Kitabchi (1987) examined the effects of Apple Classroom of Tomorrow (ACOT) involving 10 
tutors who served as both peer tutors and academic role models to reinforce the learning processes of 
African American students in an inner-city school. The study revealed that the students who participated 
in ACOT achieved higher in vocabulary, reading, language, and mathematics than the students who did 
not participate in ACOT. 

In summary, these studies found positive effects of teachers and tutors involved in CAI on the 
achievement of at-risk students, especially when their interactions with students are positive. However, 
only the work by Kestner compared matched treatment and control groups; the other studies selected their 
samples non-randomly. Therefore, the positive effects of teachers’ role in mediating students’ learning 
through computers should be viewed with caution. 

The Nature of CAI with Special Characteristics 

Although 48 achievement outcomes were examined to determine whether the type of CAI 
(drill/practice vs project/problem solving) had different effects on student achievement, most studies did 
not provide detailed information on the nature of the CAI that was implemented. Three studies reported 
special characteristics of CAI that targeted at-risk students. Willingsky and Green (1990) examined the 
effects of using desktop publishing software to enable students to cooperatively plan, write, layout, and 
produce a teen magazine. This desktop publishing program was created to provide an opportunity for low 
achievers to see themselves for the first time as writers working on an important project that would 
improve their attitudes toward writing. Despite the promising features of the desktop publishing software, 
no significant effect on improving remedial students’ achievement in writing and reading as compared to 
the control group was found. 

Two of the studies examined the analyzed the effects of CAI designed for culturally diverse 
students. Emihovich and Miller (1988) examined the effects of Logo software, which is sensitive to the 
learning styles of Afiican American students. Logo software gives students freedom to make discoveries 
on their own, promotes learning through social and cultural contexts, and encourages considerable verbal 
communications between students and teachers. The researchers found that first-grade African American 
students scored higher in mathematics than did first-grade White students under the Logo condition, while 
White students outperformed African American students in the traditional instruction condition without 
CAI. 



Wepner (1991) also examined the effects on reading and writing achievement of a software 
package that deals with the real-life issues of African American and Hispanic students as compared with 
the effects of a skill-focused software. No statistically significant differences were found in reading and 
writing achievement between the group using real-life-focused software and the group using skill-focused 
software. 

The findings from these three studies on the effects of CAI specifically designed for at-risk 
students are mixed. In addition, the quality of these studies is questionable due to their lack of control. 
More high-quality research is needed before conclusions can be made about the effects of special features 
of CAI for this population of students. 

Summary of Findings Across Studies of Computer-Assisted Instruction 

Based on the overall effect size from the meta-analysis of .37, CAI has a significantly positive 
effect on improving the achievement of at-risk students. This effect size is higher than the effect size of 
.19 computed in a previous meta-analysis of CAI involving beginning readers (including both high and 
low achievers) in the United States and abroad (Blok et al., 2002). This substantial difference in effect 
sizes may indicate that the effect of CAI is more positive for at-risk students in the United States than for 
high- and middle-level achievers outside of the United States. 

The analysis of moderating factors revealed that CAI for at-risk students is more effective in 
mathematics than in literacy. This is a new finding because previous syntheses focused on a single subject 
area, and thus did not compare the effect sizes across different subject areas (see Blok et al., 2000; Hall et 
al., 2000). The stronger effects of CAI in math may be explained by the fact that CAI can provide a rich 
visual learning environment, such as simulations, which is especially important for mathematical 
understanding. The variations in other study characteristics did not result in significantly different effect 
sizes. 



Close examination of four studies that addressed the roles of teachers/tutors and three that 
examined special nature of CAI for at-risk students revealed interesting findings. The four studies 
reported positive effects of teachers/tutors’ involvement in CAI in improving student achievement. One 
study found that the personalities of teachers affect student learning; at-risk students benefit more from 
affective teachers in learning through CAI than from non-affective teachers. The three studies that 
examined the nature of CAI yielded mixed results regarding the effects of CAI targeted for at-risk 
students. Only one study found significant effects of culturally sensitive CAI for African American 
students; the other two studies found no effects of their CAI program targeted to at-risk students. More 
high-quality studies are needed before conclusions can be drawn about the effects of teachers ’/tutors’ 
roles and the special features of CAI programs for at-risk students. 

Although this synthesis provides new and important information for policymakers and educators, 
the findings need to be interpreted with caution due to design and methodological limitations. First, the 
number of studies included in the analysis was small, and a larger set of studies might produce different 
results. Second, it cannot be assumed that only the subject area makes a difference in the effectiveness of 
CAI for at-risk students. The range of factors examined in this meta-analysis was limited since the studies 
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lacked detailed information about many contextual and moderating factors that might influence the 
observed effect sizes. For example, the amount and nature of interaction among students might make a 
difference in the effect sizes. Also, different groups of at-risk students might benefit differently from CAI 
depending on their gender and ethnicity. 

It is likely that more studies on the effectiveness of CAI will be conducted in the future as 
technology’s use and value in education increases. These studies should address factors such as student 
characteristics, learning contexts, the nature of the CAI, and types of interactions between students and 
teachers. For policymakers and practitioners to learn from these studies and implement CAI based on 
research results, it is important that researchers report the details of effective CAI programs, the contexts 
in which they work, and the characteristics of the students involved. More qualitative studies that provide 
rich contexts of CAI programs would also benefit the field, combined with well-designed quantitative 
program evaluations. These studies could provide more evidence of the benefits of CAI for at-risk 
students. 



A Final Word to Practitioners 

Computer-assisted instruction has the potential to deliver student-centered instruction and to 
provide supportive learning environments for at-risk students. The results of our meta-analysis of 17 
studies revealed that computer-assisted instruction significantly improves the achievement of at-risk 
students in mathematics and literacy. Its effects were especially strong in mathematics. CAI can be used 
to improve the achievement of at-risk students at all grade levels. Finally, both drill/practice-based CAI 
and project/problem-solving-based CAI were found to be effective in improving the achievement of at- 
risk students. 

For information on implementation of CAI and other technology-based instruction, practitioners 
should consult Roblyer and Edwards (2000) and Sandholtz, Ringstaff, and Dwyer (1997). 
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who served as both peer tutors and academic role models to reinforce the learning processes of 50 African 
American students (fifth graders) in an inner-city school. ACOT was conducted as a joint project of the 
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exposed to the JOSTENS ILS achieved significantly higher in mathematics computation, mathematics 
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Emihovich and Miller examined the effects of Logo software, which is sensitive to the learning 
styles of African American students. Logo software gives students freedom to make discoveries on their 
own, promotes learning through social and cultural contexts, and enhances considerable verbal 
communications between students and teachers. 16 first-grade African American students and 20 first- 
grade white students participated in the program under one of the three conditions: (1) Logo treatment, (2) 
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CHAPTEE. 8 



SUMMARY AND CONCLUSIONS 

Zoe Barley and Patricia A. Lauer 

This chapter provides a summary of findings and an overall conclusion about research on six 
strategies to assist low-achieving students during the school day. Results and conclusions should be 
considered in light of the methods employed, particularly the criteria for inclusion of research studies. 
Thus, conclusions discussed here are limited to the studies synthesized. 

Findings Across the Six Strategies 

Given the parameters of our literature search and study goals, the 118 studies synthesized in this 
document are representative of the research on strategies to assist low-achieving students during the 
school day. There are positive findings for the effects of each strategy on the performance of low- 
achieving students but in varying degrees, and with the exception of Peer Tutoring and Computer- 
Assisted Instruction, only a minority of the studies were coded as having high quality. 

The following describes the general results of the synthesis for each strategy, and 
Table 8.1 provides a summary. 

1 . General Instruction — Studies of instruction based on a constructivist approach (e.g., 
emphasis on meaning and understanding) found positive effects on student 
achievement. Studies of instruction based on a behaviorist approach (e.g., emphasis 
on practice and direct instruction) were of a lower research quality than the 
constructivist studies and produced mixed findings. The total number of studies on 
General Instruction was too small to support general conclusions. 

2. Cognitively Oriented Instruction — Studies of cognitive strategy instruction in 
reading, mathematics, and oral and written language found positive effects on student 
achievement and included studies coded as having high quality. These findings 
related to achievement should be considered in light of implementation findings. In 
particular, some studies found that it is important to provide opportunities for 
students to practice cognitive strategies, such as summarizing information. In 
addition, involving peers can help support students’ use of cognitive strategies. 

3. Grouping Structures — The set of studies of heterogeneous student groupings (e.g., 
cooperative learning), which include high-quality studies, found positive effects on 
student achievement. Investigators suggested the need for rigorous implementation 
and teacher training in this type of instructional grouping. Compared to studies of 
heterogeneous grouping, there was less research support for the use of ability 
grouping. Furthermore, studies of ability grouping were of lower quality. 
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4. Tutoring — In support of recent meta-analyses of tutoring, studies revealed positive 
effects on student achievement of tutoring by professionals, students, and adult 
volunteers, with some high-quality studies in the latter subcategory. Diagnostic- 
prescriptive interactions with tutees and the need for tutor training are factors that 
support implementation. 

5. Peer Tutoring — Findings from studies of peer tutoring in varied formats generally 
indicated improved student achievement in basic skills. There were two high-quality 
longitudinal studies in this set of related research. These findings should be 
considered in light of implementation findings, in particular those related to 
classroom logistics and the need to train both teachers and students in peer tutoring 
procedures. 

6. Computer-Assisted Instruction — Studies of CAI provided sufficient quantitative 
information for a meta-analysis. A majority of these studies were coded as high 
quality. Results indicate a positive effect of computer-assisted instruction on student 
achievement, which was greater for mathematics instruction than for language arts 
instruction. 



Table 8.1. Summary of Research Synthesis 



Strategy 


Type 


Number of 
High-Quality 
Studies 


Selected 

Implementation 

Considerations^ 


Synthesis Results on 
Positive Effects of 
Strategy on Student 
Achievement 


General Instruction 


Approach 


4 


Balance of direct 
instruction and authentic 
learning 


Inconclusive 


Cognitively 
Oriented Instruction 


Approach 


4 


Student practice 
Student interaction 
Teacher modeling 


Preliminary 


Grouping 

Structures 


Intervention 


4 


Heterogeneous groups 
Teacher training 
Student instructions 


Preliminary 


Tutoring 


Intervention 


5 


Tutor training 
Program monitoring 


Preliminary 


Peer Tutoring 


Intervention 


13 


Structured tasks 
Teacher monitoring 
Participant training 


Preliminary 


Computer- Assisted 
Instruction 


Intervention 


10 


Subject matter 


Conclusive 



Other implementation issues are described in the individual chapters on each strategy. 



Research Quality and Content 

Studies coded as high quality met the criteria for research validity based on standards for 
quantitative and qualitative research. Of the 118 studies synthesized, 40 studies (34 percent) were coded 
as high quality. Thirteen of the high-quality studies addressed Peer Tutoring, and 10 addressed Computer- 
Assisted Instruction. The small number of high-quality studies on four of the six strategies demonstrates 
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the need for more rigorous research designs, larger sample sizes, and more complete reporting of research 
procedures in studies of strategy effectiveness. 

Two of the strategies examined in this synthesis are approaches — General Instruction and 
Cognitively Oriented Instruction — and the other four strategies are interventions. As explained in 
Chapter 1 , approaches are conceptually more general and theoretically based, while interventions are 
more specific and applied. Studies of the two approaches were less specific than the studies of the four 
interventions. In addition, studies of interventions tended to emphasize basic skills. It may be that the 
specificity of basic interventions makes it easier for researchers to study them as compared to general 
approaches. If so, more research is needed to identify implementation details of general approaches to 
instruction so that their effects on student achievement can be tested and practitioners can have clearer 
guidelines for implementation. 

In addition, there is a need for a cumulative body of research on interventions in which studies 
systematically build on previous findings to extend results to different conditions, such as different 
student grade levels and variations in subject matter. The research studies on peer tutoring constitute an 
example of how researchers can connect their studies in ways that make the findings more informative. 
The varying research designs and their lack of connection to prior research make it difficult to give 
general recommendations to practitioners about strategy effectiveness. 

Overall Conclusion 

As indicated in Chapter 1 , this research synthesis addresses the following research question: What 
are effective strategies that can be used to assist low-achieving students? Results indicate that approaches 
using cognitively oriented instruction and interventions using heterogeneous student groupings, tutoring 
of students by adults and peers, and computer-assisted instruction can be effective strategies in improving 
the performance of low-achieving students and students who are at risk for low achievement. Although 
deriving guidance on strategy implementation is not a goal of this document, implementation issues 
emerged in the analyses of study findings. Practitioners, in particular, should be aware of the influences of 
these factors on overall strategy effectiveness and should consult the resources suggested at the end of 
each chapter for guidance. 

Finally, this synthesis was limited to studies of strategies that target low-achieving students and 
that are implemented in regular classrooms during the school day. There are other studies that examine 
classroom strategies designed to raise student achievement but that do not examine differential effects for 
low achievers. For example, Marzano’s (1998) meta-analysis identified nine instructional strategies 
(including cooperative learning, examined in Chapter 4 of the current synthesis) that improve 
achievement of the general population of K-12 students (see also Marzano, Pickering, & Pollock, 2001). 

It is possible that strategies found to work for all students might also work for the subgroup of low- 
achieving or at-risk students. However, such inferences require empirical confirmation through research 
studies designed to compare the effects of a strategy on high- versus low-achieving students. Such 
research could provide much-needed information on the most effective ways to implement instructional 
strategies that can help all students to learn. 
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APPENDIX A 



Strategies to Assist Low- Achieving Students — Coding Guide 
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A-1 . 



Coder 



Study Number 



MCREL SYNTHESIS: 2002 

Strategies to Assist Low-Achieving Students - Coding Guide 
Codes: NA = Not Applicable M=Missing 

Author(s): 

Title: Report Year 

Source: □ Journal 

□ Dissertation p ERIC report □ ERIC eval □ other 

Quality Index: Quantitative Qualitative 

Information for Table (complete after coding): Sample size Grade or age 

Student characteristics Type of comparison 

Outcome measure Direction of results* ■ 

*The number of independent samples revealing comparisons that were all positive (ap), mostly positive 
(mp), even (e) mostly negative (mn), and all negative (an) 

1. PROGRAM/INTERVENTION INFORMATION 

1.01 Determination of low-performing or at-risk: 

1 .02 School context: % FRL % LEP %M % F 

% Caucasian % African American % Latino 

% Native American % Asian %Other 

1.03 Goals within intervention strategy (optional as an aid to grouping within category) 

1 .04 Assignment of teachers to treatments: 

□ Self-selected □ Random □ Non-random □ Missing 

1.05 Program Implementer Qualifications: □ Yes DNo 

1.06 Avg. Daily Exposure hrs. Avg Weekly Exposure hrs. Tot # Weeks 

1.07 Subjects: 

□ Math □ Reading □ Writing □ Science □ Other 

1 .08 Nature of Intervention: 



2. RESEARCH DESIGN 



Student Sample Characteristics: 

2.01 □ Random 

□ Purposive 

□ Population 

□ Other 

2.02 □ Control Group(s) describe 

2.03 □ Comparison Group(s) describe 

2.04 Tot N in study 

2.04a N in Treatment Group 

2.04b N in Control Group(s) 

2.04c N in Comparison Group(s) 

2.05 Treatment Group Attrition % Control/Comparison Group Attrition % 

2.06 Grade levels in study 



2.07 


%Female 


% Male 






2.08 


% Caucasian 


%Afrcn Amer 


% Latino 


%Native Amer 




%Asian 


%Other 







Predominant Methodology (the methodology on which conclusions are based): 
2.09 □ Quantitative, quasi-experimental: 

□ One-group posttest only 

□ One-group pretest-posttest 

□ Nonequivalent groups posttest only 

□ Nonequivalent groups pretest-posttest 

□ Other 

Characteristics used for equating or matching groups: 



2.10 □ Quantitative, experimental (randomized trials ) : 

□ posttest only 

□ pretest-posttest 

□ Other 
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2.11 □ Qualitative (can check more than one) 

□ Case study 

□ Action research/Field Study 

□ Grounded theory 

□ Ethnography 

□ Other 

2.12 □ Secondary methods (describe) 



3. QUANTITATIVE ANALYSIS 



Outcome Measure Analysis 
3.01a Measure: 



Reliability reported: □ Yes □ No 

If yes, type of reliability measure 

Treatment Group 

Pretest mean sd 

Posttest mean sd 



Result of measure 



Control/Comparison group 

Pretest mean sd 

Posttest mean sd 



Unit of analysis: □ student 

Effect size 

Test statistic(s) 

3.01b Measure: 



□ class Direction of effect □ positive □ negative 



Reliability reported: □ Yes □ No 

If yes, type of reliability measure 

Treatment Group 

Pretest mean sd 

Posttest mean sd 

Unit of analysis: □ student □ class 

Effect size 

Test statistic(s) 



Result of measure 



Control/Comparison group 

Pretest mean sd 

Posttest mean sd 



Direction of effect □ positive □ negative 
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3.01c 



3.01d 



3.01e 



3.02 



3.03 



O 

ERIC 



Measure: 



Reliability reported: □ Yes 


□ No 








If yes, type of reliability measure 




Result of measure 


Treatment Group 




Control group 






Pretest mean sd 




Pretest mean 


sd 




Posttest mean sd 




Posttest mean 


sd 




Unit of analysis: □ student 


□ class 


Direction of effect □ positive 


□ negative 


Effect size 










Test statisticfs) 










Measure: 


Reliability reported: DYes 


□ No 








If yes, type of reliability measure 




Result of measure 


Treatment Group 




Control group 






Pretest mean sd 




Pretest mean 


sd 




Posttest mean sd 




Posttest mean 


sd 




Unit of analysis: □ student 


□ class 


Direction of effect □ positive 


□ negative 


Effect size 










Test statisticfs) 










Measure: 


Reliability reported: □ Yes 


□ No 








If yes, type of reliability measure 




Result of measure 


Treatment Group 




Control group 






Pretest mean sd 




Pretest mean 


sd 




Posttest mean sd 




Posttest mean 


sd 




Unit of analysis: □ student 


□ class 


Direction of effect □ positive 


□ negative 



Effect size 

Test statistic(s 

Potential for Quantitative Synthesis 



Findings/Conclusions 

What are the relevant findings/conclusions from this study that support the synthesis?: 
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3.04 Quality of Quantitative Research 

3.04a Is the information provided thorough enough for us to determine the presence of 
threats to validity? 

□ Yes □ No 

If no, go to Quality Index 3.06 and code as ’’missing.” 

3.04b Does the study represent a careful effort to eliminate threats to internal validity! 

Have the following threats been appropriately addressed and/or negated? 

□ Yes GNo 

- events that may confound treatment (history, maturation, mortality, selection 
bias) 

- use of potentially flawed measures and/or measuring techniques (pretest and 
posttest design, use of different measures in pretest and posttest, comparing gains 
of students performing on widely desperate levels, measurement error) 

- unwanted control group treatment or influence 

- the study design does not thoroughly target the research questions 

3.04c. Does the study represent a careful effort to eliminate threats to external validity! 
Have the following threats been appropriately addressed and/or negated? 

□ Yes □ No 

- interaction of treatment and subject selection or educational setting 

3.04d. Does the study represent a careful effort to eliminate threats to construct validity! 

Have the following threats been appropriately addressed and/or negated? 

□ Yes □ No 

- inadequate design of variables (definition, single measure for cause/effect) 

- anticipation and apprehension (subject or researcher knowledge of hypothesis 
potentially influenced outcome) 

- fidelity of implementation 

3.04e. Does the study represent a careful effort to eliminate threats to statistical conclusion 
validity! 

Have the following threats been appropriately addressed and/or negated? 

□ Yes □ No 

- adequate sample size to support effect (power!) 

- violated assumptions of statistical tests 

- appropriateness of statistical methods 

- regression to the mean (particularly important for our studies’ low achievers) 
statistical vs. practical significance 

3.05 Total “yes” Total “no” 
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3.06 QUALITY INDEX: 



□ Missing 

□ Low 

□ Medium 

□ High (enter index on first page) 



4. QUALITATIVE DESIGNS 

4.01 Purpose of qualitative approach: 

□ Theory building 

□ Interpretive/descriptive 

□ Other 

4.02 Data collection methods used: 

□ Nonparticipant observations 

□ Participant observations 

□ Focus groups 

□ Interviews 

□ Document review 

□ Questionnaires 

□ Other 

4.03 Data Analysis: 

□ Content analysis 

□ Constant comparative method 

□ Inductive 

□ Other 

4.04 Findings/Conclusions: 

What are the relevant fmdings/conclusions from this study that support the synthesis: 



4.05 Quality of Qualitative Research: 

Confirmability/Objectivity (the ability for others to examine all data sources and processes to 
assure that the findings are grounded in the data) 

4.05a Were any of the following used in the study? 

□ Yes (indicate which ones below) □ No 

□ Member checking 

□ Audit trail 

□ Expert/Peer review 

4.05b Did the researcher control for researcher effects? 

□ Yes (indicate how below) □ No 

□ Used unobtrusive measures 

□ Disclosed purpose of study and intentions to informants 
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□ Included variety of informants 

□ Triangulated data 

Dependability/Reliability 

(the use of sufficient methods and techniques to assure that the study’s results can be trusted) 

4.05c Are the research questions clear, and are the features of the study design congruent 
with them? 

□ Yes GNo 

4.05d Were data collected across the full range of appropriate settings, time, respondents, 
and so on suggested by the research questions? 

□ Yes DNo 

4.05e Are basic paradigms and analytic constructs clearly specified? 

□ Yes DNo 

Credibility/Internal Validity 

(the findings are credible to the reader and the researcher has used techniques to ensure the 
credibility of findings) 

4.05f Are multiple sources of evidence and/or data collection methods used to produce 
converging conclusions? 

□ Yes □ No 

If not, is there a coherent explanation for this? 

□ Yes GNo 

4.05g Were any of the following conducted? 

□ Yes (indicate which ones below) □ No 

□ Search for disconfirming evidence 

□ Generation of rival hypotheses/explanations 

□ Negative case analysis 

□ Other 

4.05h Do the presented data and measures reflect the constructs or categories of prior or 
emerging theory? 

□ Yes GNo 

Transferability/External Validity 

(the provision of sufficient “thick description” to enable the reader to decide whether the 
concepts or themes can be transferred to another setting) 

4.05i Are the characteristics of the original sample of persons, settings, processes (etc) fully 
described enough for readers to assess the potential transferability, appropriateness for 
their own settings? 

□ Yes □ No 

4.05J Does the researcher define the scope and the boundaries of reasonable generalization 
from the study? 

□ Yes □ No 

4.05k Total “yes” Total “no” 
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QUALITY INDEX: 



□ Low 

□ Medium 

□ High (enter index on first page) 
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APPENDIX B 



Meta-Analysis Methods 

Since all 17 studies examined the effects of CAI by the comparison of achievement score 
between treatment and control groups, we computed the d-index of the effect size (Cooper 1998; Hedges 
& Olkin, 1985). The overall effect size was computed by subtracting the effect size at the time of the 
pretest from the effect size at the time of the posttest in order to accurately estimate the effect size of CAI 
(see Blok et al., 2002). The pooled standard deviation was used to reflect the different standard errors and 
sample sizes between the treatment and the comparison groups (Hedges & Olkin, 1985). For the studies 
without pre-test scores, the effect size from the posttest was used as the overall effect size. Five studies 
provided t values or F values without mean scores and/or standard error. T values and F values were 
converted to d indexes based on the equations presented in Cooper (1998). 

In computing the overall effect size, we employed the fixed model (Cooper, 1998). The effect 
size from each study was weighted by the estimate of sample size based on the general assumption that 
studies with larger sample sizes produce more reliable estimates of the effect size. Homogeneity analysis 
was conducted in order to determine whether the effect sizes from the selected studies vary more than 
expected by sampling error alone. If the result is significant, which means that the effect sizes vary across 
studies more than expected by the sampling error alone, we should seek the moderating factors that 
explain the variation across studies. Results of homogeneity analysis indicated that the Q statistics is 
significant; therefore, we proceeded with further analysis of identifying the moderators of the effect sizes. 

The strategy of shifting the unit of analysis (Cooper, 1998) was utilized when we moved from 
calculating the overall effect size to identifying the moderators of the effect sizes. In computing the 
overall effect size, we disaggregated each study to an independent sample unit. While some studies report 
outcomes based on a single sample, other studies report their results for multiple independent samples. 

For example, a study may report math score gains of fourth and fifth graders separately. In this case, this 
study provides two independent samples and their outcomes. However, another study may report forty 
fifth graders’ achievement gain in math and reading separately. In this case, the mean effect size from 
math and reading achievement gains was computed so that this single group of samples has only one 
effect size. The number of independent samples in a study varied from one to five. 

When we examined the relationship between effect sizes and publication characteristics such as 
publication year and type, we employed each study as the unit of analysis. When we moved to the 
analysis of identifying moderators related to experimental characteristics, we disaggregated each study to 
the smallest unit the study could provide. In this way, we identified whether the effect sizes depend on the 
samples’ grade level and/or the subject area. The number of the effect sizes each study provides varied 
from one to twenty. 
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